- ListSupportedNetworkServiceProvidersCmd will now return Providers along with its element's services and boolean 'canEnableIndividualServices' that indicates if for this Provider services can be enabled/disabled
- add & update NetworkServiceProvider changed to take in the list of services to enable. While adding a provider, if list is null then all services supported by the element are enabled by default.
- ListNetworkServices enhanced to take in a provider name and returns services of that specific provider.
* can be specified for Shared network only
* if not specified for the Shared networks, try to locate it based on the zoneId and tags. If tags is not null, pick up first physicalNetwork from the zone that has matching tags. If tags is null, and there are none/more than 1 physical netwroks in the zone, error out.
As DhcpElement/VirtualRouterElement/RedundantVirtualRouterElement is decided to
be the service provider of the physical network, this API should be called to
add a new element, with correlated network service provider ID.
Then e.g. ConfigureVirtualRouterElementCmd should be called to configure and
enable the element.
Added PortForwardingServiceProvider, StaticNatServiceProvider, rename
PasswordServiceProvider to UserDataServiceProvider(may rename to a better name
later).
Add related function for service providers.
DHCP range, domain name, etc. are the property of network, not virtual router
specific.
The focus of virtual router configuration would on separate enable/disable each
service it provided.
2)Re-apply all existing firewall rules as a part of implement call. TODO: Cleanup all existing rules from the backend (leave them in the DB) as a part of shutdown call
1. able to restore VM from its original template
2. Only allow to restore when VM is running or stopped
3. after restoring, VM state will not change, e.g. running is still running
status 9949: resolved fixed
TODO:
- Still leaving the provider columns in data_center schema as-is for CloudKit and BareMetal
- ExternalNetworkDeviceMgrImpl still needs to fix the dataCenter.setProviders calls and externalNetworkApplicance usage checks to see if zone has external networking.
Limitations:
* can't upgrade to the network offering with lesser number of services
* can upgrade only when the service provider of the original offering is not external (domR, dhcp, elb) to the external type of the provider
-Bringing add/delete/list of all external network devices under one unified API's (addNetworkDevice, deleteNetworkDevice, listNetworkDevice)
-Refactoring External network manager to work from both sets of API's add/delete/list NetworkDevice and add/delete/list External Firewall/LoadBalancer
- Make all API commands Async and add events
- Make BroadcatsDomainRange case insensitive
- Process all _networkElements to build the Service -> Provider map during NetworkMgr::configure()
Add configure command for these virtual router based elements. The commands
should be different for different elements.
The context of configuration would be added later.
- Create Zone changes and changes to data_center table to remove vlan, securityGroup fields
- Physical Network lifecycle APIs
- Physical Network Service Provider APIs
- DB schema changes
* moved all services to the separate table, map them to the network_offering+provider.
* added state/securityGroupEnabled properties for the networkOffering
* added ability to list by state/securityGroupEnabled in listNetworkOfferings api command
2) New service: SourceNat
Changes:
- Added a new interface 'PluggableService'
- Any component that can be packaged separately from cloudstack, can implement this interface and provide its own property file listing the API commands the component supports
- As an example have made VirtualNetworkApplianceService pluggable and a new configureRouter command is added
- ComponentLocator reads all the pluggable service from componentLibrary or from components.xml and instantiates the services.
- As an example, DefaultComponentLibrary adds the pluggable service 'VirtualNetworkApplianceService'
- Also components.xml.in has an entry to show how a pluggable service can be added, but it is commented out.
- APIServer now reads the commands for each pluggable service and when a command for such a service is called, APIServer sets the required instance of the pluggable service in the coomand.
- To do this a new annotation '@PlugService' is added that is processed by APIServer. This eliminates the dependency on the BaseCmd to instantiate the service instances.
Since we would introduce a way to specify each service provider in the network
offering, it's better for redundant virtual router as a separate service
provider.
Also isRedundant() flag in the network offering would be removed. Redundant
virtual router temporality won't work from now. Until we're able to add
different network elements/service providers in network_offering.
In the past, the NetworkElement would cover almost all the functionality that
e.g. virtual router can cover: firewall, source NAT, static NAT, password,
VPN... So anyone want to implement the NetworkElement would have to implement
these service's specific methods, even it wouldn't support it. Also, if we want
to find a e.g. FirewallServiceProvider, we have to proceed all the current
network service providers, to call a method to know if it support such service.
That's neither elegant nor scaling way to do it.
As the first step, this patch separates each ServiceProvider from NetworkElement
(there are some interface already out of NetworkElement, so this patch slightly
modifies them too), and only the class would implement the correlated interface, would
have the ability to do these services.
Changes:
- Added a new API 'migrateSystemVm' backed by MigrateSystemVMCmd.java to migrate system VMs (SSVM, consoleproxy, domain routers(router, LB, DHCP))
- This is Admin only action
- The existing API 'migratevirtualmachine' is only for user VMs
Otherwise they can't be deserialized.
Fix the following exception on KVM host:
ERROR [agent.transport.Request] (Agent-Handler-2:null)
Caught problem with
[{"ClusterSyncCommand":{"_interval":60,"_skipSteps":20,"_steps":0,"_clusterId":2,"contextMap":{},"wait":0}}]
com.google.gson.JsonParseException: The JsonDeserializer
com.cloud.agent.transport.ArrayTypeAdaptor@44ed904 failed to deserialized json
object
[{"ClusterSyncCommand":{"_interval":60,"_skipSteps":20,"_steps":0,"_clusterId":2,"contextMap":{},"wait":0}}]
given the type class [Lcom.cloud.agent.api.Command;
...
Caused by: java.lang.RuntimeException: No-args constructor for class
com.cloud.agent.api.ClusterSyncCommand does not exist. Register an
InstanceCreator with Gson for this type to fix this problem.
at
com.google.gson.MappedObjectConstructor.constructWithNoArgConstructor(MappedObjectConstructor.java:64)
...
1) Introduce new managers - ProjectManager and DomainManager. Moved all domain related code from AccountManager to DomainManager.
2) Moved some code from ManagementServerImpl to the correct managers.
3) New resource limit for Domain - Project
Added two New values "all" and "default" to global config "network.loadbalancer.haproxy.stats.visibility" . With this change, it can take six possible value:
global - stats visible from public network.
guest-network - stats visible only to guestnetwork.
link-local - stats visible only to link local network(for xen and kvm).
disabled - stats disabled.
all - stats available on public,guest and link-local. (Newly added)
default - stats availble on the serving http port, this does need any specific http port.(Newly added)
Except "default" and "disabled", all the rest of 4 need to configure the stats port.
Force stop the router would release all the resources it used, but router may
still running. Add a column "stop_pending" in the database, and stop it when the
router come back.
Admin would able to choose to force destroy such router, then recover the
network using restartNetwork command with cleanup=false.
status 11036: resolved fixed
1) Use row locks instead of global lock when update resource_count table. When update resource_count for account, make sure that we lock account+all related domains
2) Insert resource_count records for account/domain at the moment when account/domain is created.
3) As a part of DB upgrade, insert missing resource_count records for all non-removed accounts/domains
Conflicts:
core/src/com/cloud/alert/AlertManager.java
server/test/com/cloud/agent/MockAgentManagerImpl.java
clean up tests for security group manager v2
move interval to listener -- allows it to be configurable if needed
fix mocks
Enhanced logging for security group manager (from zucchini)
fix merge issues
merge issues
Changes :
- Fixing API doc +response name + errorMessage
- Adding seperate events to Egress rules
- Egress rules Using the same database table as that of ingress with new column type.
Pending Tasks:
- db upgrade
- database table rename from security_ingress_rule to generic name, renaming some of the jave class from ingress to generic name.
- Retesting on kvm
Changes:
To make sure migration does not attempt to pick a host that has running VMs more than the max guest VM's limit:
- Changed manual migration to call host allocators to return a list of hosts suitable for migration. Host allocators check for the max guest VM limit.
- Earlier we returned hosts with enough capacity but now Host Allocators make other checks along with capacity. So the list of hosts returned are hosts that have enough capacity AND satisfy all other conditions like host tags, max guests limit etc. Or in other words Allocators dont return the hosts that dont satisfy all conditions even if they have capacity.
-Therefore, now we mark the list of hosts returned for manual migration as 'suitable' hosts instead of 'hasenoughCapacity' in the HostResponse.
- HA migration already calls allocators, so no change is needed there.
Changes:
- Adding a new table 'hypervisor_capabilities' that will record capabilities for each hypervisor version. Added db schema changes for this.
- Currently a few capabilities have been added, namely, 'max_guests_limit' and 'security_group_enabled'
- Added a new column 'hypervisor_version' to host table. StartupRouting command now takes in this parameter. It should be set when a host connects.
- If a host's hypervisor version is not present, we find all the capabilities rows for that hypervisor type and use the first record.
- 'max_guests_limit' is the maximum number of running guest Vms that a host can have for the given hypervisor.
- Host Allocators use this limit and skip a host if the number of running VMs on that host exceeds this limit.
- adding supprt for Netscaler VPX & MPX load blancers
- implemented for virtual networking
- works only with new fetched public IP, inline support is not added yet
Changes:
- Changes to updateHostCmd to accepts hosttags parameter
- Changes to wipe out existing tags and save new ones in host_tags DB.
- UpdateHost is Admin only operation - so only root admin can update host tags
Changes:
- CreateTemplate and RegisterTemplate now support adding a template tag. It is a string value. This is root-admin only action - only admin can add template tags.
- ListTemplates will return the template tag in response.
- HostAllocator changed to use template tag along with the existing tag on service offering. If both tags are present, allocator now finds hosts satisfying both tags. If no hosts have both tags, allocation will fail.
- DB changes to add new column to vm_template table.
- DB upgrade changes for upgrade from 2.2.10 to 2.2.11
status 11029: resolved fixed
Commit also includes the following:
* map firewall rule to pf/lb/staticNat/vpn when the firewall rule is created as a part of pf/lb/staticNat/vpn rule creation
* when delete firewall rules, also delete related firewall rule
1) On enableStaticNat command we actually send the command to the backend (we used to just upgrade the DB in the past). The backend command carries sourceIp and destIp, and creates IP to IP mapping on the domR.
2) On disableStaticNat for the Ip address in addition to cleaning up port ranges, we also delete IP to IP mapping on the domR.
1) Added new apis: createFirewallRule, deleteFirewallRule, listFirewallRules
2) Modified existing apis - added boolean openFirewall parameter to createPortForwardingRule/createIpForwardingRule/createRemoteAccessVpn. If parameter is set to true, open firewall on the domR before creating an actual PF rule there
Modified backend calls appropriately.
3) Schema changes for firewall_rules table:
* startPort/endPort can be null now
* added icmp_type, icmp_code fields (can be not null only when protocol is icmp)
4) Added new manager - FirewallManagerImpl
Conflicts:
api/src/com/cloud/api/BaseCmd.java
client/tomcatconf/commands.properties.in
server/src/com/cloud/api/ApiResponseHelper.java
server/src/com/cloud/configuration/DefaultComponentLibrary.java
server/src/com/cloud/network/lb/LoadBalancingRulesManagerImpl.java
server/src/com/cloud/network/rules/RulesManagerImpl.java
1) Added new apis: createFirewallRule, deleteFirewallRule, listFirewallRules
2) Modified existing apis - added boolean openFirewall parameter to createPortForwardingRule/createIpForwardingRule/createRemoteAccessVpn. If parameter is set to true, open firewall on the domR before creating an actual PF rule there
Modified backend calls appropriately.
3) Schema changes for firewall_rules table:
* startPort/endPort can be null now
* added icmp_type, icmp_code fields (can be not null only when protocol is icmp)
4) Added new manager - FirewallManagerImpl
implement pool-wise VM sync,
For XenServer, VM fullSync is pool-wise now, VM deltaSync is still per host
Conflicts:
server/src/com/cloud/vm/VirtualMachineManagerImpl.java
Load non-routing resource in ClusteredAgentManager includes External DHCP, PxeServer, ExternalFirewall, ExternalLoadBalancer
Bug 9887 - baremetal: support for image operation (create template from guest disk) (edit)
changes in line with UI
Load non-routing resource in ClusteredAgentManager includes External DHCP, PxeServer, ExternalFirewall, ExternalLoadBalancer
Bug 9887 - baremetal: support for image operation (create template from guest disk) (edit)
changes in line with UI
put copy scripts in SetupCommand,
1. initiate returns host version,
2. if it doesn't match with DB, update DB, and reconnect the host.
status 9997: resolved fixed
The step to upgrade xenserver,
1. put cluster in Unmanaged state through UI , then MS will not talk to hosts in the cluster
2. upgrade xenserver according to XenServer upgrade guide.
3. put cluster in Managed state through UI, then MS will reconnect hosts
TODO,
1. UI
2. vm pool sync , leveraged from kelven's work
The step to upgrade xenserver,
1. put cluster in Unmanaged state through UI , then MS will not talk to hosts in the cluster
2. upgrade xenserver according to XenServer upgrade guide.
3. put cluster in Managed state through UI, then MS will reconnect hosts
TODO,
1. UI
2. vm pool sync , leveraged from kelven's work
Part 2
commit 797839360c65cd348d2eb20630521177ab0919de
bug 9154: redundant virtual router
commit 8ff7f230204d4d3a7a4adee75523a9a84f4276fe
bug 9154: Replace domain_router.is_master with domain_router.redundant_state in DB
commit 230b99e9e0b152648f1dd2a5eab6f22315b8e7b4
bug 9154: Add redundant state to DomainRouterResponse
commit ccefb5ff5e83d713798a347c99bce1a0d04b4317
bug 9154: Add router fault state report
commit 7a3090378f9785caecf741b70554f6ea17c41764
bug 9154: Send alert if found two virtual routers in master state
commit 66831056e4bf27665871bccd24e6159071564847
bug 9154: Code clean up
commit bf3f58a85741fa7118bd848a42d8b21baa4478d4
bug 9154: Add isRedundantRouter to DomainRouterResponse
Part 1
This backport contained:
commit 52317c718c25111c2535657139b541db0c9d1e1f
bug 9154: Initial check in for enabling redundant virtual router
commit 54199112055d754371bfb141168fb5538bf6d6ea
Add host verification for CheckRouterCommand
commit cef978a228c90056ead9be10cbc4de74c2b8de76
Fix CheckRouterAnswer's isMaster report
commit 4072f0a6991ac3b63601a1764fbe14188965f62f
Some build fixes and code refactoring for redundant router
commit 4d3350b7cd8ee2706a9bace4437fc194e36c8dd5
Redundant Router: Fix OVS
commit 6a228830e7c46d819fa0c3317e159e041337e887
Fix findByNetwork()/findByNetworkAndPod()'s return
commit c627777b3d5bdbcd60db4032cebd349a5b1ecd83
Redundant Router: Fix isVmAlive()
commit e1275d2514adc41f8744f5107d4069c38be195f1
Only issue CheckRouterCommand to redundant routers
And all modification to the scripts till
commit 4e3942462ed3fde3a3d7011e95839e2128fba514
logging changes
in the master branch.
Kelven Yang
Sheng Yang
alena
prachi
Alex Huang
Edison Su
anthony
Abhinandan Prateek
root
frank
kishan
Nitin
Edison Su
Murali Reddy
frank
Chiradeep Vittal
Edison Su
Naredula Janardhana Reddy
keshav
Alex Huang
Jessica Wang