* Add Netris module and Add netris provider
* Fix
* Add Netris Provider to the zone creation wizard
* add steps to zone wizard for adding netris controller and public traffic
* cleanup
* Add missing config key
* Add routed mode offerings for Netris VPC (#3)
* Add routed mode offerings for Netris VPC
* update vpc offering name
* generalize the offering creation method for network providers
* log
* remove debug log
* fix failing build
* Add dependency and Netris API client (#4)
* Add dependency and first approach to Netris API client
* Fix authentication and create Netris API client, in progress sites listing
* Fix get sites
* Support for listing VPCs (#5)
* List tenants
* Delegate API classes creation to the SDK and simply invoke the desired API class through CloudStack (#7)
* Delegate API classes creation to the SDK and simply invoke the desired API class through CloudStack
* Pass default auth scheme for now
* Support adding netris provider to CloudStack and Netris VPC Creation (#6)
* Support adding netris provider to CloudStack
* revert marvin change
* add license and perform session check when provider is added
* add license and remove unused import
* fix build failure - uunused imports
* address comments
* fix provider name
* add Netris network element
* add license
* Add netris management APIs and netris service provider
* add license
* revert change
* remove other network elements from Netris element
* fix api name in doc generator
* remove logs
* move session alive check to CheckHealthCommand exec
* Fix zone creation wizard to configure netris provider
* Upgrade GSON version - from PR 8756
* Add additional parametes to the add Netris provider API
* add netris as a host
* add additional params to the resoponse and update UI
* Rename site to site_name
* Create Netris VPC (#8)
* Delegate API classes creation to the SDK and simply invoke the desired API class through CloudStack (#7)
* Delegate API classes creation to the SDK and simply invoke the desired API class through CloudStack
* Pass default auth scheme for now
* Drop for_nsx and for_tungten columns in favour of checking the provider on the ntwserviceprovider map table
* Remove missing setForTungsten occurrence
* Remove forNsx from VPC offerings
* Create Netris VPC
* Fix VPC offerings listing and remove unused dao
* Create VPC fixes
* Upgrade GSON version - from PR 8756
* Fix VPC creation response by using the latest SDK code
* Fix unit test
* Remove unused import
* Fix NSX unit tests after refactoring
* Add Netris key to the VLAN Details table (#10)
* Add Netris key to the VLAN Details table
* update for_<provider> column to be generic
* Fix VPC and add IPAM allocation for the VPC CIDR (#9)
* Fix VPC and add IPAM allocation for the VPC CIDR
* Remove VPC logic
* Use zoneId accountId and domainId on resources creation
* Fix naming
* Fix VR public nic issue
* Fix Netris Public IP for VPC source NAT allocation
* Add Netris VPC Subnets and vNets (#11)
* Add Netris VPC Subnets and vNets
* fix compilation errors
* Add netris subnet
* refactor naming convention to differentiate between VPC tiers and Isolated networks
* revert marvin change
* fix constructor - build failure
* Add support to filter netris offerings, delete netris provider when zone is being deleted
* Fix build
* Fix VPC creation
* Fix vnet creation
* unnecesary log
---------
Co-authored-by: nvazquez <nicovazquez90@gmail.com>
---------
Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>
---------
Co-authored-by: nvazquez <nicovazquez90@gmail.com>
* Fix unit tests
* Add support to delete VNets and Subnets (#13)
* Add support to delete VNets and Subnets
* Add support to delete vnet resources
* Add support to delete vnet resources
* extract code to method
---------
Co-authored-by: nvazquez <nicovazquez90@gmail.com>
* Add missing suffix return (#14)
* Set up Netris Public range on new zone addition (#15)
* Set up Netris Public range on new zone addition
* Add dependency to calculate subnet containing a start and end IP
* Remove unused import
* Move dependency to the netris module
* Rename Netris IP range
* Refactor logic
* Revert "Refactor logic"
This reverts commit 7ec36a81320444c37e7bb914dd895060b663411b.
* Fix setup range after adding Netris Provider
* Fix VXLAN range adding on zone creation
* Pass VXLAN ID during creation of Netris vNets (#16)
* add zone params to accepts management vnet
* Release vxlan associated to the netris broadcast domain type
* handle update network broadcast uri
* Update Subnet purpose for Netris Public Traffic (#17)
* Update Subnet purpose for Netris Public Traffic
* search for existing subnet of common purpose type
* Fix VR Public IP address (#20)
* Fix VR Public IP address
* Do not set the Public IP range on Netris side that is not part of the Netris IP Public Pool
* Leave only systemvms tag for the first element
* Fix NSX compatibility
* Pass network gateway instead of network CIDR for Netris vNet creation (#21)
* Run moodifyvxlan script if broadcast domain type is Netris (#18)
* Add support to create Netris VPC / Network offerings (#22)
* Add support to create Netris VPC / Network offerings
* fix support services for netris provider type
* Phase4 - Add support for Source NAT, Static NAT and Port Forwarding (#19)
* Run moodifyvxlan script if broadcast domain type is Netris
* Add Netris NAT offerings
* Add support to add Source nat rules for Natted offering
* fix api params while creating Netris source NAT rule
* Add support to add and delete source nat rule on netris
* Add support to create /32 NAT subnet
* Add support to add and delete Static NAT rules in Netris (#23)
* Add support to add and delete Static NAT rules in Netris
* fix static nat creation on netris & removal of subnet on deletion of static nat rule
* remove nat subnet after deltion of the static nat rule
* add check to see if subnet already exists and add license header
* Add port forwarding rules as DNAT rules in Netris (#24)
* Add port forwarding rules as DNAT rules in Netris
* Fixes
* Allow removing DNAT rules
* Fixes
* Fix subnet search
* Fix update SNAT only for SNAT rules
* Address comments
* Fix
* Fix netris pom xml
* Fix SNAT rule creation
* Fix IP and port placements (#27)
* Fix IP and port placements
* fix dnat to IP for PF rules
* change dnatport
---------
Co-authored-by: Nicolas Vazquez <nicovazquez90@gmail.com>
* List only Netris Public IPs for NAT operations (#26)
* List only Netris Public IPs for NAT operations
* rename getter and change type
* fix failing unit tests
* list all IPs if forProvider is not passed
* fix list public IPs for external providers with additional IP range
* filter provider Ips in a zone with external provider setup
* Prevent acquiring IP that is not from the external provider range
* formating
---------
Co-authored-by: nvazquez <nicovazquez90@gmail.com>
* Support to pass provider when creating public ip range and create IPAM on Netris (#28)
* UI: support to pass provider when creating public ip range
* prevent adding public ip range for a provider that isnt supported in zone
* Create public range on Netris when created on CloudStack
---------
Co-authored-by: nvazquez <nicovazquez90@gmail.com>
* Revert UI filtration for public IPs (#29)
* Fix issue with pagination of public addresses listed after filtering for external providers
* Revert UI filteration for public IPs for external network provider enabled zones
* Fix unit tests (#30)
* Add Netris Tag parameter to the Network provider and fix zone creation wizard (#33)
* Add Netris Tag parameter to the Network provider
* remove unused import
* Fix public IP ranges creation on zone creation (#34)
* use single quotes
---------
Co-authored-by: Nicolas Vazquez <nicovazquez90@gmail.com>
* Fix SourceAddress for SNAt to VPC cidr (#35)
* Fix VPC/network offering service list for external network providers in Routed mode (#32)
* Fix network offering service list for external network providers in Routed mode
* filter out unsupported services based on network mode
* fix supported services list for vpc offering for external providers in Routed mode
* Add support to add and delete and update static routes on Netris (#37)
* Add support to add static routes in Netris
* support to delete static routes on netris
* add defensive check for nextHop
* Add support to update static routes
* add state
* pass empty list for switched to avoid timeout
* Netris: search static route by name and next hop if exists
---------
Co-authored-by: Wei Zhou <weizhou@apache.org>
* Netris FR1b: Support Remote Access VPN and Site-to-Site VPN in VPC VR (#41)
* Static Routes: support nexthop
* Update api/src/main/java/org/apache/cloudstack/api/command/user/vpc/CreateStaticRouteCmd.java
Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>
* PR#10064 VR: apply iptables rules when add/remove static routes
* PR#10065 UI: fix cannot open 'Edit tags' modal for static routes
* PR#10066 Static Routes: fix check on wrong global configuration
* PR#10067 VR: fix site-2-site VPN if split connections is enabled
* PR#10081 server: do not allocate nic on public network for NSX VPC VR
* PR#10082 UI: create VPC network offering with conserve mode
* PR#10083 VR: allow outgoing traffic from RAS/VPN clients
* PR#10086 server: fix typo removeaccessvpn in VirtualRouterElement
* server: Add check on Public IP for remote access VPN
* Revert "PR#10083 VR: allow outgoing traffic from RAS/VPN clients"
This reverts commit 2f9b9f428947cac91de322fbdf4a980902a1c0a0.
* VPC: fetch same used IP for domain router if VR is not Source NAT
* VR: pass has_public_network to VR and configure RA/S2S VPN left peers
* Revert "PR#10081 server: do not allocate nic on public network for NSX VPC VR"
This reverts commit 809e269ed6b361d9df1fcef6537762c5612863e0.
* VPC: fetch same used IP for domain router if VR is not Source NAT (v2)
* VR: fix /etc/hosts and nameservers in dnsmasq.conf if VPC VR is not guest gateway
prior to this PR
```
root@r-1167-VM:~# cat /etc/hosts
127.0.0.1 localhost
127.0.1.1 r-1167-VM
::1 localhost ip6-localhost ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
172.21.1.33 dummy-vpc-vpn-001
172.21.1.1 r-1167-VM data-server
root@r-1167-VM:~# cat /etc/dnsmasq.d/cloud.conf
dhcp-hostsfile=/etc/dhcphosts.txt
listen-address=127.0.0.1,172.21.1.234
dhcp-range=set:interface-eth1-0,172.21.1.234,static
dhcp-option=tag:interface-eth1-0,15,cs2cloud.internal
dhcp-option=tag:interface-eth1-0,6,172.21.1.1,10.0.32.1,8.8.8.8
dhcp-option=tag:interface-eth1-0,3,172.21.1.1
dhcp-option=eth1,26,1500
dhcp-option=tag:interface-eth1-0,1,255.255.255.0
```
the lines should be
```
172.21.1.234 r-1167-VM data-server
dhcp-option=tag:interface-eth1-0,6,10.0.32.1,8.8.8.8
```
* server: Enable static NAT for Domain router if it is not Source NAT
* server: Enable static NAT for Domain router on UI
* server: assign Public IP to VPC VR and enable static nat if VR is not Source NAT
* server: configure dns1 if VR is not Source NAT
* server: remove check on Firewall service when list network service providers
* UI: remove dot from message.enabled.vpn
* systemvm: add default route via first guest gateway if VR does not have public IP/interface
* VR: add fw_dhcpserver for shared network
* VR: pass has_public_network to VR and configure RA/S2S VPN left peers (v2)
* UI: fix request error when create a VPC tier in a non-Netris/NSX env
* systemvm: add default route via first guest gateway (v2)
* VR: configure iptables rules for S2S vpn on first guest interface
* VR: allow FORWARD to guest interfaces if VR is not Public
* VR: configure remote access vpn on first guest interface if not public
* VR: fix error 789 in RA VPN client when both RA and S2S are configured
* server: Apply Static Route for RA/S2S VPN in VPC VR
* VR: do not set mark for Public interface when VR is not really public
* VPN: do not disable static nat if it is used by a RA/S2S VPN
* server: skip check on network conserve mode if disable/enable RA VPN on Router IP
* server: set forRouter to false when release a IP
* VR: diable IP spoofing protection on default guest network
* VR: fix iptables rules only when only S2S vpn is enabled
* UI: show 'VPN Connections' section
* VPC: new methods to configure/reconfigure Static NAT for VPC VR
* API: set Type in ip address response to DomainRouter if it is used by VR
* server: do not allow IP release if it is used by RA or S2S VPN gateway
* VR: check if interface is added
* VR: add default route only when ip is associated to first guest interface
* VR: fix ipsec conf for l2tp and s2s vpn
* server: save placeholder IP for VPC VR to fix the new VR IP when vpc tier is auto-shutdown
* server: get non-placeholder NIC for VPC VR
* VR: wait 15 seconds after starting password server
* server: fix unable to configure static nat due to 'invalid virtual machine id'
* UI: fix link of router in info card
* VPC: apply static route for VPC VPN if needed (refactoring)
* server: fix VR IP of first VPC tier is the VM gateway
* server: update or remove all existing static routes when shutdown a network
* server: update ipaddress after disabling static nat to fix vpc deletion issue
* servr: disable remote access VPN as part of VPC dstroy
* server: apply static routes when implement a vpc tier
* server: apply static routes even if next hop is null
* server: fix Cannot invoke "com.cloud.vm.NicProfile.getRequestedIPv4()" because "requested" is null
* Netris: Update Vpn provider to VpcVirtualRouter
* Netris: Add Vpn service to network offerings and networks
* server: fix CIDR of VPN ip range
* server: set isVrGuestGateway by SoureNat/Gateway service with Provider.VPCVirtualRouter
* VR: password server takes 10-15 seconds to start if VR IP is not configured in /etc/hosts
* Netris: add back routesPutBody.setStateStatus
* engine/schema: remove SQL changes in schema-41910to42000.sql
---------
Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>
* Add support for Gateway service for Netris VPC and network offerings (#39)
* Add support for Gateway service for Netris VPC and network offerings
* Restore UserData service
* add gateway only to vpc service
* Add support for gateway service for external network providers for networks in routed mode
* add support for gateway svc
* Revert "add support for gateway svc"
This reverts commit 06645cd1c6d08a81ede5d1431497ea3f2efdc5dc.
* Fix VPC offering creation
* Fix VR public NIC after Gateway service is set to Netris
---------
Co-authored-by: nvazquez <nicovazquez90@gmail.com>
* Netris VPN: Fix s2s vpn status update and isolated network implementation (#42)
* server: fix NPE when deploy vm on isolated network
* vpn: fix s2s vpn status is not updated
Prior to this fix
```
java.lang.IllegalArgumentException: Class com.cloud.agent.api.CheckS2SVpnConnectionsAnswer declares multiple JSON fields named 'details'; conflict is caused by fields com.cloud.agent.api.CheckS2SVpnConnectionsAnswer#details and com.cloud.agent.api.Answer#details
at com.cloud.agent.transport.ResponseTest.testCheckS2SVpnConnectionsAnswer(ResponseTest.java:42)
```
* test: fix test_01_vpn_usage as now it is only possible to create VPN on Source NAT if it uses VR
* VR: fix unable to create remote access VPN on regular isolated network
the error is
```
File "/opt/cloud/bin/configure.py", line 1242, in process
self.remoteaccessvpn_iptables(self.dbag['public_interface'], public_ip, self.dbag[public_ip])
~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
KeyError: 'public_interface'
```
* Release NAT IP subnet when VPC is removed or IP is released (#44)
* Release NAT IP subnet when VPC is removed or IP is released
* add license
* Add support to add IPv6 Public IP range as IPAM Allocation / Subnet on Netris (#36)
* Add support to add IPv6 Public IP range as IPAM Allocation / Subnet on Netris
* Add ipam alloc and subnet for the ipv6 subnet associated to the vpc tier network
* remove commented code
* Phase5 - Support for ACLs in Netris (#31)
* Add support for Netris ACLs
* acl support
* Make acl api call to netris to create the rule
* refactor add acl rule to populate the right fields
* support icmp type acl rule
* acl rule creation - move netrisnetworkRule
* Update ACL naming on Netris
* Add support for Deletion of netris acls
* Add support to delete and re-order ACL rules
* support creation of default acl rules and replacing acl rules
* fix NSXNetworkRule
* Add global routing flag on subnet creation (#45)
* Support change snat ip (#46)
* Support updating VPC Source NAT IP
* Optimize code
* Update source NAT IP
* Fix naming convention for NAT subnets to follow other resources (#47)
* Fix naming convention for NAT subnets to follow other resources
* Use vpc ID for nat subnets
* Use new nat subnet name for deletion of static nat rule
* fix naming convevntion for nat subnet
* Keep Vpn service to default VPC offering with Natted mode only (#50)
* Add Vpn service to default VPC offering with Routed mode
* Revert change on VPC offering and fix VPN service only for Netris NAT mode
* Validate if given CIDR belongs to a bigger allocation in Netris before creating the zone-level allocation (#48)
* Validate if given CIDR belongs to a bigger allocation in Netris before creating
* rename method
* Phase5 - Support for LB - create, delete and Update operations (#49)
* Add support for Netris ACLs
* acl support
* Make acl api call to netris to create the rule
* refactor add acl rule to populate the right fields
* support icmp type acl rule
* acl rule creation - move netrisnetworkRule
* Update ACL naming on Netris
* Add support for Deletion of netris acls
* Add support to delete and re-order ACL rules
* support creation of default acl rules and replacing acl rules
* fix NSXNetworkRule
* Fix naming convention for NAT subnets to follow other resources
* Use vpc ID for nat subnets
* Phase5 - Support for LB - create, delete and Update operations
* Use new nat subnet name for deletion of static nat rule
* add support to add netris lb rule
* support deletion of LB rule on Netris
* add checks when editing unsupported fields of LB rule for Netris and hide columns on the UI
* fix test failure
* fix imports
* add license
* address comments
* Enable Autoscaling on Netris for CPU and memory (#51)
* Enable Autoscaling on Netris for CPU and memory
* Fix monitor autoscale group and cleanup
* Rename autoscaling group method
* Integrate Autoscaling by allowing to update LB rules
* Refactor according to the SDK changes
* Fix the test failures noticed on #44 (#52)
* Increase code coverage (#54)
* Increase code coverage
* More unit tests
* Remove credentials and mock api client
* NetrisResource tests
* Fix unit test
* Add support to add and remove ACL rules when CIDR list is passed when creating LB rules (#53)
* Add support to add and remove ACL rules when CIDR list is passed when creating LB rules
* add deny all rule
* delete the deny rule as well
* Fix build (#57)
* Prevent Index Out of Bounds exception when naming IPAM subnets (#58)
* Prevent Index Out of Bounds exception when naming IPAM subnets
* fix linter
* Delete netris IPv6 subnet (#59)
* Netris VPN: add static route when update a non-existent static route (#60)
* Fix VPC tier creation failure - prevent creating IPv6 IPAM allocation if it already exists (#61)
* Update netris VPC and tier name (#56)
* Update netris VPC and tier name
* add support to update vpc tier name
* add license
* support editing names of dual stack VPCs
* VR/server: configure default gateway and RA/S2S VPN on the IP/interface with minimum network_id (#43)
* server: fix NPE when deploy vm on isolated network
* vpn: fix s2s vpn status is not updated
Prior to this fix
```
java.lang.IllegalArgumentException: Class com.cloud.agent.api.CheckS2SVpnConnectionsAnswer declares multiple JSON fields named 'details'; conflict is caused by fields com.cloud.agent.api.CheckS2SVpnConnectionsAnswer#details and com.cloud.agent.api.Answer#details
at com.cloud.agent.transport.ResponseTest.testCheckS2SVpnConnectionsAnswer(ResponseTest.java:42)
```
* test: fix test_01_vpn_usage as now it is only possible to create VPN on Source NAT if it uses VR
* VR: fix unable to create remote access VPN on regular isolated network
the error is
```
File "/opt/cloud/bin/configure.py", line 1242, in process
self.remoteaccessvpn_iptables(self.dbag['public_interface'], public_ip, self.dbag[public_ip])
~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
KeyError: 'public_interface'
```
* VR/server: configure default gateway and RA/S2S VPN on the IP/interface with minimum network_id
* Don't add deny rule if no CIDR list is passed (#62)
* Hide the Stickiness Configure button for Netris Load Balancers (#72)
* Update IPAM subnet purpose to nat before NAT operations if its different (#71)
* Netris VPN: create vpc gateway with specified IP (#63)
* Netris: fix UnsupportedOperationException when create VPC offering with NATTED mode (#75)
fixes
```
2025-03-21T10:42:55,039 ERROR [c.c.a.ApiServer] (qtp1513608173-21:[ctx-f9c7f002, ctx-bcfe846d]) (logid:e12e798f) unhandled exception executing api command: [Ljava.lang.String;@3a1416cd java.lang.UnsupportedOperationException
at java.base/java.util.AbstractList.add(AbstractList.java:153)
at java.base/java.util.AbstractList.add(AbstractList.java:111)
at org.apache.cloudstack.api.command.admin.vpc.CreateVPCOfferingCmd.getServiceProviderMapForExternalProvider(CreateVPCOfferingCmd.java:248)
```
* [UI] Zone wizard creation improvements - rename hostname to url and remove port for Netris Provider (#77)
* [UI] Zone wizard creation improvements - rename hostname to url and remove port for Netris Provider
* Fix schema column for url instead of hostname
* Fix Static NAT rules naming (#83)
* Netris: create VPN gateway with specified public IP on UI (#82)
* Netris vpn: apply static routes when start or delete a VPN connection (#85)
* Netris VPN: apply static routes when start S2S VPN
* Netris: list static routes and revoke the routes which are not needed
* Netris: use route name (x.x.x.0/x) instead of prefix (x.x.x.0) and get clean cidr list
* Netris VPN: fix NPE when list static routes
* Update plugins/network-elements/netris/src/main/java/org/apache/cloudstack/service/NetrisApiClientImpl.java
* Delete IPv6 allocation after tier removal on VPC with dual stack offering (#86)
* Netris pass v6 gateway (#87)
* pass v6 gateway to netris
* pass v6 gateway to netris
* refactor to address comments
* remove imports
* [VR] Fix IPv6 NIC IP on the VR (#89)
* Add support to edit ACL rules (#74)
* Add support to edit ACL rules
* add support to update acl rules
* remove test file
* VR: advertise SLAAC prefix only if VR is gateway (#91)
* Make reorder ACL items invoke Netris controller (#90)
* VR: fix radvd misconfiguration for non-netris env (#92)
* [VR] Fix object comparisson to string comparisson on python (#93)
* Fix unit tests for ACL (#94)
* Use the previously assigned vNet for Netris Network when it transitions to Implemented state after gc (#88)
* Use the previously assigned vNet for Netris Network when it transitions to Implemented state after gc
* Fix unit tests
---------
Co-authored-by: nvazquez <nicovazquez90@gmail.com>
---------
Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>
Co-authored-by: Wei Zhou <weizhou@apache.org>
* Make volume attachment disk controller selection consistent with VM creation and start
* Update vmware-base/src/main/java/com/cloud/hypervisor/vmware/util/VmwareHelper.java
Co-authored-by: dahn <daan.hoogland@gmail.com>
* Choose disk controllers after converting osdefault
* Rename function
---------
Co-authored-by: dahn <daan.hoogland@gmail.com>
This introduces the multi-arch zones, allowing users to select the VM arch upon deployment.
Multi-arch zone support in CloudStack can allow admins to mix x86_64 & arm64 hosts within the same zone with the following changes proposed:
- All hosts in a clusters need to be homogenous, wrt host CPU type (amd64 vs arm64) and hypevisor
- Arch-aware templates & ISOs:
- Add support for a new arch field (default set of: amd64 and arm64), when unspecified defaults to amd64 and for existing templates & iso
- Allow admins to edit the arch type of the registered template & iso
- Arch-aware clusters and host:
- Add new attribute field for cluster and hosts (kvm host agents can automatically report this, arch of the first host of the cluster is cluster's architecture), defaults to amd64 when not specified
- Allow admins to edit the arch of an existing cluster
- VM deployment form (UI):
- In a multi-arch zone/env, the VM deployment form can allow some kind of template/iso filtration in the UI
- Users should be able to select arch: amd64 & arm64; but this is shown only in a multi-arch zone (env)
- VM orchestration and lifecycle operations:
- Use of VM/template's arch to correctly decide where to provision the VM (on the correct strictly arch-matching host/clusters) & other lifecycle operations (such as migration from/to arch-matching hosts)
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* Create/Export OVA file of the VM on external vCenter host, to temporary conversion location (NFS)
* Fixed ova issue on untar/extract ovf from ova file
"tar -xf" cmd on ova fails with "ovf: Not found in archive" while extracting ovf file
* Updated VMware to KVM instance migration using OVA
* Refactoring and cleanup
* test fixes
* Consider zone wide pools in the destination cluster for instance conversion
* Remove local storage pool support as temporary conversion location
- OVA export not possible as the pool is not accessible outside host, NFS pools are supported.
* cleanup unused code
* some improvements, and refactoring
* import nic unit tests
* vmware guru unit tests
* Separate clone VM and create template file for VMware migration
- Export OVA (of the cloned VM) to the conversion location takes time.
- Do any validations with cloned VM before creating the template (and fail early).
- Updated unit tests.
* Check conversion support on host before clone vm / create template on vmware (and fail early)
* minor code improvements
* Auto select the host with instance conversion capability
* Skip instance conversion supported response param for non-KVM hosts
* Show supported conversion hosts in the UI
* Skip persistence map update if network doesn't exist
* Added support to export OVA from KVM host, through ovftool (when installed in KVM host)
* Updated importvm api param 'usemsforovaexport' to 'forcemstodownloadvmfiles', to be generic
* Updated hardcoded UI messages with message labels
* Updated UI to support importvm api param - forcemstodownloadvmfiles
* Improved instance conversion support checks on ubuntu hosts, and for windows guest vms
* Use OVF template (VM disks and spec files) for instance conversion from VMware, instead of OVA file
- this would further increase the migration performance (as it reduces the time for OVA preparation / archiving of the VM files into a single file)
* OVF export tool parallel threads code improvements
* Updated 'convert.vmware.instance.to.kvm.timeout' config default value to 3 hrs
* Config values check & code improvements
* Updated import log, with time taken and vm details
* Support for parallel downloads of VMware VM disk files while exporting OVF from MS, and other changes below.
- Skip clone for powered off VMs
- Fixes to support standalone host (with its default datacenter)
- Some code improvements
* rebase fixes
* rebase fixes
* minor improvement
* code improvements - threads configuration, and api parameter changes to import vm files
* typo fix in error msg
* Veeam: find storage pool by path for PreSetup and VMFS
* Veeam: support VMware distributed virtual switch
* Veeam: sync volumes on Solidfire after backup restoration
user faced the issue that backup is restored but the DATA disk is gone (ROOT disk is ok)
```
2024-05-03 12:00:32,868 ERROR [o.a.c.b.BackupManagerImpl] (API-Job-Executor-13:ctx-aa8a1d85 job-149661 ctx-73328567) (logid:6510cf06) Failed to import VM [vmInternalName: i-169-9679-VM] from backup restoration [{"backupType":"Full","externalId":"821ca400-a5da-4282-bf3f-7c7e38a6cdb4","id":257,"uuid":"69399101-5cbd-461c-8a48-f0c70eac0b24","vmId":9679}] with hypervisor [type: VMware] due to: [Couldn't find storage pool -iqn.2010-01.com.solidfire:3p53.data-9679.221-0].
```
On managed storage, the datastore name of DATA disk is determined by the iscsi_name of the volume.
* Veeam: set correct path for DATA disks on solidfire
* 4.19:
protect against null-path (#8915)
UI: Fix missing locale strings for Status widget (#8792)
Add a shutdownhook to remove jobs owned by the process (#8896)
* 4.18:
protect against null-path (#8915)
UI: Fix missing locale strings for Status widget (#8792)
Add a shutdownhook to remove jobs owned by the process (#8896)
* NSX integration - skeletal code
* Fix module not loading on startup
* add upgrade path and daos
\n add nsx controller command
* add support for adding and listing nsx provider to a zone
* add license
* add default VPC offering and update upgrade path
* add global setting to enable nsx plugin
* add delete nsx controller operation
* add nsxresource
* add NSX resource , api client, create tier1 gw
* update db
* update response and add license
* Add support to create and delete nsx tier-1 gateway
* add license
* cleanup and add skeletal code for network creation
* add create/delete segment and UI integration
* add license
* address code smells - part 1
* fix test / build failure
* NSX integration - skeletal code
* Fix module not loading on startup
* add upgrade path and daos
\n add nsx controller command
* add support for adding and listing nsx provider to a zone
* add license
* add default VPC offering and update upgrade path
* add global setting to enable nsx plugin
* add delete nsx controller operation
* add nsxresource
* add NSX resource , api client, create tier1 gw
* update db
* update response and add license
* Add support to create and delete nsx tier-1 gateway
* add license
* cleanup and add skeletal code for network creation
* add create/delete segment and UI integration
* add license
* address code smells - part 1
* fix test / build failure
* add ui changes + update nsx_provider table transport zones + use NSX broadcast domain for add nics to router
* ui: fix password field, and backend changes
* add route advertisement
* update offering
* update offering
* add sleep before deletion of vpc / tier g/w for ports to be removed
* move creation of segments to design phase
* change provider to VPC router for Dhcp & dns service in an nsx offering
* Add public nic for NSX
* reserve first IP (after g/w) of subnet for router nic - NSX
* revert reserving 1st IP in vpc segments
* [NSX] Create a DHCP relay and add it to a VPC tier segment (#107)
* Create DHCP relay command and execute request
* In progress integrate with networking
* Create DHCP relay config on the network VR allocation
* Revert domain router dao changes
* Create DHCP relay con VR nic plug to NSX network
* Link DHCP relay config to segment after creation
* [NSX] Cleanup DHCP Relay config on segment deletion (#108)
* Cleanup DHCP Relay config on segment deletion
* update segment & relay name generators and call delete dhcprelay after deletion of segment
* address comment
* [NSX] Fix DHCP relay config deletion was missing zone name (#8068)
* [NSX] Refactor API wrapper operations (#8059)
* [NSX] Refactor API wrapper operations
* Big refactor
* Address review comment
* change network cidr to cidr to prevent NPE
* add domain and zone names to the various networks - vpc & tier
---------
Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>
* Nsx unit tests (#8090)
* Add tests
* add test for NsxGuestNetworkGuru
* add unit tests for NsxResource
* add unti tests for NsxElement
* cleanup
* [NSX] Refactor API wrapper operations
* update tests
* update tests - add nsxProviderServiceImpl test
* add unit test - NsxServiceImpl
* add license
* Big refactor
* Address review comment
* change network cidr to cidr to prevent NPE
* add domain and zone names to the various networks - vpc & tier
* fix tests
---------
Co-authored-by: nvazquez <nicovazquez90@gmail.com>
* modify NSX resource naming convention (#8095)
* modify NSX resource naming convention
* remove unused imports
* add a setup phase between desgin and implementation of a network for intermediary steps
* add method to all classes
* NSX: Refactor Network & VPC offering (#8110)
* [NSX] Refactor API wrapper operations
* Network offering changes for NSX
* fix services and provider combination
* address comments: rename param
* update nsx_mode parameter
---------
Co-authored-by: nvazquez <nicovazquez90@gmail.com>
* fix test
* [NSX] Allow NSX isolated networks (#8132)
* Add network offerings for NSX on isolated networks
* Fix offerings creation
* In progress NSX isolated network
* Fixes
* Fix NIC allocation to router
* NSX: Add Step for Adding Public traffic network for NSX During zone creation (#8126)
* NSX: Add Step for Adding Public traffic network for NSX
* address comments and cleanup
* address comment
* remove indent
* NSX: Create and Delete static NAT & Port forward rules (#8131)
* NSX: Create and delete NSX Static Nat rules
* fix issues with static nat
* add static nat
* Support to add and delete Port forward rules
* add license
* fix adding multiple pf rules
* cleanup
* fix lint check
* fix smoke tests
* fix smoke tests
* Nsx add lb rule (#8161)
* NSX: Create and delete NSX Static Nat rules
* fix issues with static nat
* add static nat
* Support to add and delete Port forward rules
* add license
* fix adding multiple pf rules
* cleanup
* NSX: Add support to create and delete Load balancer rules
* fix deletion of lb rules
* add header file and update protocol detail
* build failure fix
* [NSX] Add SNAT support (#8100)
* In progress add source NAT
* Fix after merge
* Fix tests
* Fix NPE on isolated network deletion
* Reserve source NAT IP when its not passed for NSX VPC
* Create source NAT rule on VR NIC allocation
* Fix update VPC and remove VPC to update and remove SNAT rule
* Fix packaging
* Address review comment
* Fix build
* fix build - unused import
* Add defensive checks
* Add missing design to NSX public guru
---------
Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>
* NSX: Fix VR public NIC allocation (#8166)
* NSX: fix LB member addition and deletion and add defensive checks (#8167)
* Fix public NIC NPE on broadcast URI
* NSX: Router Public nic to get IP from systemVM Ip range (#8172)
* NSX: Router Public nic to get IP from systemVM Ip range
* Fix VR IP address and setSourceNatIp command
* NSX: hide systemVM reserved IP range SourceNAT
* fix test
---------
Co-authored-by: nvazquez <nicovazquez90@gmail.com>
* fix test failure
* test failure fix
* [NSX] Fix update source NAT IP (#8176)
* [NSX] Fix update source NAT IP
* Fix startup
* Fix API result
* NSX - add LB route Advertizement (#8192)
* [NSX] Add ACL types support (#8224)
* NSX: Create segment group on segment creation
* Add unit tests
* Remove group for segment before removing segment
* Create Distributed Firewall rules
* Remove distributed firewall policy on segment deletion
* Fix policy rule ID and add more unit tests
* Fix DROP action rules and transform tests
* Add new ACL rules
* Fixes
* associate security policies with groups and not to DFW and add deletion of rules
* Fix name convention
---------
Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>
* NSX: Fix creation of VPCs (#8320)
* Fix ACL rules creation (#8323)
* [NSX] Fix database views (#8325)
* NSX: Add CKS Support & Firewall rules for Isolated Networks (#8189)
* NSX: Add ALL LB IP to the list of route advertisements in tier1
* NSX: Support Source NAT on NSX Isolated networks
* NSX: Cks Support
* NSX: Create segment group on segment creation
* Add unit tests
* Remove group for segment before removing segment
* Create Distributed Firewall rules
* Remove distributed firewall policy on segment deletion
* Fix policy rule ID and add more unit tests
* Add support for routed NSX Isolated networks \n and non RFC 1918 compliant IPs
* Add support for routed NSX Isolated networks \n and non RFC 1918 compliant IPs
* Add Firewall rules
* build failure - fix unit test
* fix npes
* Add support to delete firewall rules
* update nsx cks offering
* add license
* update order of ports in PF & FW rules
* fix filter for getting transport zones
* CKS support changed - MTU updated, etc
* add LB for CKS on VPC
* address comments
* adapt upstream cks logic for vpc
* rever mtu hack
* update UI changes as per upstream fix
* change display test for CKS n/w offerings for isolated and VPC tiers
* add extra line for linter
* address comment
* revert list change
---------
Co-authored-by: nvazquez <nicovazquez90@gmail.com>
* fix ui build failure
* [NSX] Address SonarCloud Bugs (#8341)
* [NSX] Address SonarCloud Bugs
* Fix NSX API connection issues
* NSX: Add unit tests to increase coverage (#8355)
* NSX: Add unit tests
* cleanup unused imports
* add more unit tests
* add tests for publicnsxnetworkguru
* add license
* fix build failures
* address sonar comment
* fix security hotspots
* NSX: Add more unit tests (#8381)
* NSX : Unit tests
* remove unused imports
* remove unused import causing build failure
* fix build failures due to unused imports
* fix build failure
* fix test assertion
* remove unused imports
* remove unused import
* Nsx UI zone bug (#8398)
* NSX: Attempt to fix NSX Zone creation bug for public networks
* fix zone wizard public traffic issue
* add proper filtering of offerings based on VPC nsx mode
* clean up console logs
* NSX: Fix code smells and reported bugs (#8409)
* NSX: Fix code smells and reported bugs
* fox override issue
* remove unused imports
* fix test
* refactor code to reduce complexity
* add lisence
* cleanup
* fix build failure
* fix build failure
* address comments
* test - add config to ignore certain files from test coverage
* test exclusion of classes from test cov
* rever pom changes
* [NSX] Add more unit tests (#8431)
* [NSX] Add more unit tests
* More tests
* Fix build errors
* NSX: Prevent creation of L2 and Shared networks for NSX (#8463)
* NSX: Prevent creation of L2 and Shared networks for NSX
* add checks to backend to prevent creation of l2 and shared networks in nsx zones and filter only nsx offerings when creating isolated networks
* cleanup
* NSX: Fix code smells (#8436)
* NSX: Fix code smells
* Add changes to service creation logic
* CKS: Add action to during firewall rule creation (#8498)
* NSX,UI: Deduplicate network list when creating kubernetes clusters (#8513)
* NSX: Make LB service selectable in network offering (#8512)
* NSX: Make LB service selectable in network offering
* fix label
* address comments
* address comments
* NSX: Add appropriate error message when icmp type is set to -1 for NSX (#8504)
* NSX: Add appropriate error message when icmp type is set to -1 for NSX
* address comments
* update text
* fix test
* fix test - build failure
* fix test - build failure
* NSX: Cleanup NSX resources during k8s cluster cleanup (#8528)
* fix test failure
* NSX: Improve segment deletion process (#8538)
* NSX: Add passive monitor for NSX LB to test whether a server is available (#8533)
* NSX: Add passive monitor for NSX LB to test whether a server is available
* Add active monitors too
* fix build failure
* NSX: Add check for ICMP code / type for NSX zones (#8542)
* NSX: Fix Routed Mode for Isolated and VPC networks (#8534)
* NSX: Fix Routed Mode for Isolated and VPC networks
* NSX: Fix Routed mode - add checks for ports added for FW rules
* clean up code
* fix build failure
* NSX: Add retry logic with sleep to delete segments (#8554)
* NSX: Add retry logic with sleep to delete segments
* add logs
* NSX: Fix custom ACL check (#2)
* NSX: Fix custom ACL check
* NSX: Fix custom ACL check
* Nsx vpc routed mode (#5)
* NSX: Fix VPC routed mode
* NSX: VPC route mode
* remove unnecessary changes
* Nsx: Support internal LB (#4)
* NSX: Support internal LB service in NSX
* add lb removal logic
* Fix UI issue hiding internal LB tab
* Refactor method name
---------
Co-authored-by: nvazquez <nicovazquez90@gmail.com>
* NSX: Improve NSX resource cleanup process (#3)
* Fix unit test
* NSX: Add SourceNAT service to the default Routed offering for VPC (#13)
* Fix VPC restart with cleanup (#12)
* NSX: Fix ACL rule removal on replacement and fix rule order (#11)
* NSX: fix smoke test failure for ACLs (#9)
* Fix unit tests
* Fix NSX plugin pom XML
* NSX: Add support to re-order ACL rules (NSX FW rules) (#14)
* [WIP] NSX: Add support to re-order ACL rules (NSX FW rules)
* fix reordering of acl rules on all networks that it is associated to
* clean up and attempt test fix
* Fix tests
* Remove unused import
* tweak reorder logic
---------
Co-authored-by: nvazquez <nicovazquez90@gmail.com>
* Fix zone creation issue for internal load balancer
* Fix
* Fix unit test
* fix logger
* fix logger
* fix logger
* NSX: Fix VPC form to ignore source NAT IP when creating VPCs and fix label
* Move SQL changes to the newest schema file
* NSX: Last Fixes
* Fix build
---------
Co-authored-by: nvazquez <nicovazquez90@gmail.com>
* Normalize logs
All classes that could have their loggers inherited from their fathers had their own loggers deleted;
Most loggers didn't have to be static, so most of them were normalized so that they wouldn't be;
All loggers are protected now;
Static logger's name are now 'LOGGER';
Non-static logger's name are now 'logger';
New class DbUpgradeAbstractImpl created so that all Upgraders extend it and inherit its logger
* Upgrade log4j
* fix errors caused by the merge
* Refactor cglibThrowableRenderer functionality to log4j2 and upgrade the last configuration files
* fix sonarcloud bug
* Fix errors caused by merge, remove some unused loggers, and rename a variable that was mistakenly renamed on the normalization commit
* Readd snmpTrapAppender, remove TestAppender
* Regenerate changes
* regenerate changes
* refactor last custom appender
* fix systemvm configuration xml
* Regenerate changes
* Regenerate changes
* regenerate changes
* Regenerate changes
* regenerate changes
* regenerate changes
* regenerate changes
* Fix utils pom
* fix some tests
* regenerate changes
* Fix jar being printed on exception
* fix logging in system VMs, fix commands not having log4j2 classpath.
* regenerate changes
* Fix some unwanted renomeations
* fix end of file
* regenerate changes
* regenerate changes
* fix merge error
* regenerate changes
* fix tests
* regenerate changes
* regenerate changes
* regenerate changes
* regenerate changes
* regenerate changes
* regenerate changes
* regenerate changes
* readd reload4j to tungsten as juniper depends on it
* Regenerate changes
* regenerate changes
* regenerate changes
* regenerate changes
* regenerate changes
* re-add reload4j dependency to network-contrail, as juniper depends on it
* regenerate changes
* regenerate changes
* regenerate changes
* fix typo
* regenerate changes
* regenerate changes
* Fix end of files
* regenerate changes
* add logj42 to cloud-utils-SHADED.jar
* regenerate changes
* regenerate changes
* regenerate changes
* regenerate changes
* regenerate changes
* regenerate changes
* regenerate changes
* regenerate changes
* Regenerate changes
* Regenerate changes
* Regenerate changes
* regenerate changes
* Regenerate changes
* regenerate changes
* Regenerate changes
* Regenerate changes
* Regenerate changes
* regenerate changes
* Regenerate changes
* Regenerate changes
* fix some tests
* Regenerate changes
* Regenerate changes
* fix test
* Regenerate changes
* Regenerate changes
* veeam: detach only the restored volume during backup restore
Steps to reproduce the issue
1. create a VM (A) with ROOT and DATA disk
2. assign to a backup offering
3. create backup
4. create another VM (B)
5. restore the DATA disk of VM A, and attach to VM B
6. When operation is done, check the datastore
Without this change, the ROOT image is not removed and left over on the datastore.
```
[root@ref-trl-5933-v-Mr8-wei-zhou-esxi2:/vmfs/volumes/5f60667d-18d828eb] ls -l /vmfs/volumes/5f60667d-18d828eb/CS-RSTR-dfb6f21c-a941-49db-9963-4f0286a17dac
total 1784840
-rw------- 1 root root 5242880000 Jan 24 09:23 ROOT-722_2-flat.vmdk
-rw------- 1 root root 499 Jan 24 09:23 ROOT-722_2.vmdk
```
With this change, the whole temporary vm has been destroyed.
```
[root@ref-trl-5933-v-Mr8-wei-zhou-esxi2:/vmfs/volumes/5f60667d-18d828eb] ls -l /vmfs/volumes/5f60667d-18d828eb/CS-RSTR-734bee3b-640c-4ff0-a34b-bc45358565b2
ls: /vmfs/volumes/5f60667d-18d828eb/CS-RSTR-734bee3b-640c-4ff0-a34b-bc45358565b2: No such file or directory
```
* veeam: fix wrong disk size in debug message
* veeam: sync backup repository after operations are done
got exception of some operations which succeeds due to the following error
```
2024-01-19 10:59:52,846 DEBUG [o.a.c.b.v.VeeamClient] (API-Job-Executor-42:ctx-716501bb job-4373 ctx-2359b76d) (logid:b5e19a17) Veeam response for PowerShell commands [PowerShell Import-Module Veeam.Backup.PowerShell -WarningAction SilentlyContinue;$restorePoint = Get-VBRRestorePoint ^| Where-Object { $_.Id -eq '1d99106a-b5c8-4a1e-958d-066a987caa5f' };if ($restorePoint) { Remove-VBRRestorePoint -Oib $restorePoint -Confirm:$false;$repo = Get-VBRBackupRepository;Sync-VBRBackupRepository -Repository $repo;} else { ; Write-Output 'Failed to delete'; Exit 1;}] is: [^M
Restore Type Job Name State Start Time End Time Description ^M
------------ -------- ----- ---------- -------- ----------- ^M
ConfResynchronize Configuration Dat... Starting 19/01/2024 10:59:52 01/01/1900 00:00:00 ^M
^M
^M
Remove-VBRRestorePoint : Win32 internal error "Access is denied" 0x5 occurred while reading the console output buffer. ^M
Contact Microsoft Customer Support Services.^M
At line:1 char:196^M
+ ... orePoint) { Remove-VBRRestorePoint -Oib $restorePoint -Confirm:$false ...^M
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^M
+ CategoryInfo : ReadError: (:) [Remove-VBRRestorePoint], HostException^M
+ FullyQualifiedErrorId : ReadConsoleOutput,Veeam.Backup.PowerShell.Cmdlets.RemoveVBRRestorePoint^M
^M
].
```
* veeam: fix unable to detach volume when restore backup and attach to vm then detach the volume
It also happened when destroy the original or backup VM
```
2024-01-24 10:10:03,401 ERROR [c.c.s.r.VmwareStorageProcessor] (DirectAgent-74:ctx-95b24ac7 10.0.35.53, job-25995/job-25996, cmd: DettachCommand) (logid:7260ffb8) Failed to detach volume!
java.lang.RuntimeException: Unable to access file [de52fdd3386b3d67b27b3960ecdb08f4] i-2-723-VM/7c2197c129464035bab062edec536a09-flat.vmdk
at com.cloud.hypervisor.vmware.util.VmwareClient.waitForTask(VmwareClient.java:426)
at com.cloud.hypervisor.vmware.mo.DatastoreMO.moveDatastoreFile(DatastoreMO.java:290)
at com.cloud.storage.resource.VmwareStorageLayoutHelper.syncVolumeToRootFolder(VmwareStorageLayoutHelper.java:241)
at com.cloud.storage.resource.VmwareStorageProcessor.attachVolume(VmwareStorageProcessor.java:2150)
at com.cloud.storage.resource.VmwareStorageProcessor.dettachVolume(VmwareStorageProcessor.java:2408)
at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.execute(StorageSubsystemCommandHandlerBase.java:174)
at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.handleStorageCommands(StorageSubsystemCommandHandlerBase.java:71)
at com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareResource.java:589)
at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)
at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2024-01-24 10:10:03,402 INFO [c.c.h.v.u.VmwareHelper] (DirectAgent-74:ctx-95b24ac7 10.0.35.53, job-25995/job-25996, cmd: DettachCommand) (logid:7260ffb8) [ignored]failed to get message for exception: Unable to access file [de52fdd3386b3d67b27b3960ecdb08f4] i-2-723-VM/7c2197c129464035bab062edec536a09-flat.vmdk
```
* vmware: create restored volume with new UUID and attach to VM
This PR fixes several issues in the testing of Veeam 11 and Veeam12
- Import Veeam.Backup.PowerShell and silently ignore the warning messages
- Fix issue when assign vm to backup offerings, which caused by separator (\r\n)
- Fix authorization failure in veeam 12a, which is because v1_4 is not supported in veeam 12a any more
- Fix exception if backup name has space
- Fix backup metrics in veeam12, which is because powershell command does not return the values needed
- Fix Incorrect datetime value, which is because powershell command returns a datetime which is not supported in Java
- Fix issue during backup restoration if VM has both ROOT and DATA disks.
This PR also has the following update
- Add integration test test/integration/smoke/test_backup_recovery_veeam.py
- Make some UI changes
- Add zone setting backup.plugin.veeam.version. If it is not set, CloudStack will get veeam version via powershell commands.
- Add zone setting backup.plugin.veeam.task.poll.interval and backup.plugin.veeam.task.poll.max.retry
This PR adds the capability in CloudStack to convert VMware Instances disk(s) to KVM using virt-v2v and import them as CloudStack instances. It enables CloudStack operators to import VMware instances from vSphere into a KVM cluster managed by CloudStack. vSphere/VMware setup might be managed by CloudStack or be a standalone setup.
CloudStack will let the administrator select a VM from an existing VMware vCenter in the CloudStack environment or external vCenter requesting vCenter IP, Datacenter name and credentials.
The migrated VM will be imported as a KVM instance
The migration is done through virt-v2v: https://access.redhat.com/articles/1351473, https://www.ovirt.org/develop/release-management/features/virt/virt-v2v-integration.html
The migration process timeout can be set by the setting convert.instance.process.timeout
Before attempting the virt-v2v migration, CloudStack will create a clone of the source VM on VMware. The clone VM will be removed after the registration process finishes.
CloudStack will delegate the migration action to a KVM host and the host will attempt to migrate the VM invoking virt-v2v. In case the guest OS is not supported then CloudStack will handle the error operation as a failure
The migration process using virt-v2v may not be a fast process
CloudStack will not perform any check about the guest OS compatibility for the virt-v2v library as indicated on: https://access.redhat.com/articles/1351473.
This PR adds new functionality to copy snapshots across zones and take snapshots for multiple zones.
Copy functionality is similar to template copy. The source zone acts as the web server from where the destination zone(s) can download the snapshot files. For this purpose, a new API - `copySnapshot` has been added. The response for copySnapshot will be returning zone and download details from the first destination zone of the request. This behaviour is similar to the `copyTemplate` API.
In a similar manner, multiple zones can be selected while taking the snapshots or creating snapshot policies. For this snapshot will be taken in the base zone(in which volume is present) and then copied to the additional zones. A new parameter - `zoneids` has been added to `createSnapshot` and `createSnapshotPolicy` APIs.
As snapshots can be present on multiple zones (secondary stores), a new parameter `zoneid` has been added to delete the snapshot copy on a specific zone.
`listSnapshots` API has been updated to allow listing snapshot entries for different zones/datastores. New parameters - `showUnique`, `locationType` have been added.
Events generated during snapshot operations will now be linked to the snapshot itself rather than the volume of the snapshot.
`listSnapshotPolicies` and `createSnapshotPolicy` APIs will return zone details of the zones in which backup will be scheduled for the policy.
----
New API added
`copySnapshot`
Request and response params updated for APIs
```
- listSnapshots
- deleteSnapshot
- createTemplate
- listZones
- listSnapshotPolicies
- createSnapshotPolicy
```
UI updated for
- Snapshot detail view
- Create snapshot form
- Create snapshot policy form
- Create volume (from snapshot) form
- Create template (from snapshot) form
Doc PR: https://github.com/apache/cloudstack-documentation/pull/344
PR: https://github.com/apache/cloudstack/pull/7873
This fixes the following cases in which Solidfire storage integration
caused issues when using Solidfire datadisks with VMware:
1. Take Volume Snapshot of Solidfire data disk
2. Delete an active Instance with Solidfire data disk attached
3. Attach used existing Solidfire data disk to a running/stopped VM
4. Stop and Start an instance with Solidfire data disks attached
5. Expand disk by resizing Solidfire data disk by providing size
6. Expand disk by changing disk offering for the Solidfire data disk
Additional changes:
- Use VMFS6 as managed datastore type if the host supports
- Refactor detection and splitting of managed storage ds name in storage
processor
- Restrict storage rescanning for managed datastore when resizing
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Nicolas Vazquez
João Jandre
Fabricio Duarte
Abhishek Kumar
Suresh Kumar Anaparti
Suresh Kumar Anaparti
John Bampton
Vishesh
Wei Zhou
Daan Hoogland
Pearl Dsilva
GaOrtiga
Harikrishna
Rohit Yadav