mirror of https://github.com/apache/cloudstack.git
44 Commits
| Author | SHA1 | Message | Date |
|---|---|---|---|
Vishesh
|
0655075f51
|
Feature: Forgot password (#9509)
* Feature: Forgot password * Address comments * fixups * Make forgot password disabled by default * Apply suggestions from code review * Address comments |
|
Wei Zhou
|
679ce1a639
|
feature: Dynamic and Static Routing (#9470)
This PR contains 3 features - IPv4 Static Routing (Routed mode) #9346 Design document: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=306153967 - AS Numbers Management #9410 Design Document: https://cwiki.apache.org/confluence/display/CLOUDSTACK/BGP+AS+Numbers+Management - Dynamic routing Design Document: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=315492858 - Document: https://github.com/apache/cloudstack-documentation/pull/419 Rename nsx mode to routing mode by ``` git grep -l nsx_mode |xargs sed -i "s/nsx_mode/routing_mode/g" git grep -l nsxmode |xargs sed -i "s/nsxmode/routingmode/g" git grep -l nsxMode |xargs sed -i "s/nsxMode/routingMode/g" git grep -l NsxMode |xargs sed -i "s/NsxMode/RoutingMode/g" ``` - re-organize sql changes - fix NPE as rules do not have public ip - fix missing destination cidr in ingress rules - disable network usage for routed network - fix DB exception as network_id is -1 during network creation - apply ingress/egress routing rules - VR changes to configure nft rules for isolated network - VR: setup nft rule for control network - VR: flush all iptables rules - fix NPE which is because ingress rules do not have public ip associated - fix dest cidr is missing in nft tables - add ip4 routing and ip4 routes to list network and list vpc response - fix ingress rule is missing when vr is restarted - fix icmp types in nft rules - add tab to manage routing firewall rules - fix ingress rules are not applied when VR is restarted - add default rules in FORWARD chain - fix create vpc offerings - fix public ip is not assigned to vpc - fix network offering is not listed when create vpc tier - add is_routing to boot args of vpc vr - remove table ip4_firewall in vpc vr - release or remove subnet when remove a network - implemenent fw_vpcrouter_routing - fix wrong ip familty when flush ipv4 rules - fix acl rules are not applied due to wrong version (should be 6 which means ip6 rules are removed) - add default rules for vpc tiers so that tcp connections (e.g. ssh) work - append policy rules after default rules - remove /usr/local/cloud/systemvm/ in routers - throw an exception when allocate subnet with cidrsize - fix some TODOs - add new parameters to update API - return type Ipv4GuestSubnetNetworkMap when get or create subnet - fix firewall rules are broken - add domain_id and account_id to db - add domain/account/project to ipv4 subnet response - create ipv4 subnet for domain/account/project - check conflict when update ipv4 subnet - ui changes - add parent subnet to response - add list for ipv4 subnet - implement some methods - fix list subnets for guest networks by zoneid - UI changes - fix delete ipv4 subnet for network - fix ipv4 subnet is set to zone guest network cidr if cidrsize is specified - add zone info to response if parent subnet is null but network is not - fix gateway/cidr is not set when create network with cidrsize - fix order of nft rules in the VRs * Routed v24 - add classes in marvin base.py * Routed v25 - add test_01_subnet_zone - fix dedicate to domain/account failure - list subnets for network by keyword and subnet * Routed v26: implement subnet auto-allocation - add utils for split ip ranges into small subnets - add utils to get start/end ip of a cidr - implement subnet auto-generation - add global settings * Routed 27: add subnet for VPC - add db column for vpc_id - add db record for vpc - remove db record when delete a vpc - add checkConflicts methods - remove duplicated settings - check ipv4 cidr when create subnet * Routed v28: update smoke tests - update test_ipv4_routing.py - search subnets by networkid * Routed 29: fix vpc and add more tests - fix createnetwork in vpc - add vpc id/name to response - fix zone id/name are not displayed in some cases - add smoke test for vpc - add smoke tests for failed cases - add smoke test for connectivity checks - marvin: add "-q" to ssh command * Routed 31: ui and smoke tests - UI: add link to network in list view - add nftables rules check in VRs * Routed 32: add chain OUTPUT and more rules - fix the issue 80/443/8080 is not reachable from VR itself ``` 2024-06-27 10:21:52,121 INFO Executing: systemctl start cloud-password-server@172.31.1.1 2024-06-27 10:21:52,128 INFO Service cloud-password-server@172.31.1.1 start 2024-06-27 10:21:52,129 INFO Executing: ps aux 2024-06-27 10:24:02,175 ERROR Failed to update password server due to: <urlopen error [Errno 110] Connection timed out> ``` * Routed: fix dns search from VMs in Isolated networks * Routed: fix VPC dns issue due to gateway IP is missing in cloud.conf This is caused by NSX integration, and fixed by https://github.com/apache/cloudstack/pull/9102/ * Routed: rename routing_mode to network_mode * Routed: replace centos5.5 template in smoke test as dhclient does not work in the vms // this does not work refer to https://dominikrys.com/posts/disable-udp-checksum-validation/#ignoring-udp-checksums-with-nftables and https://forum.openwrt.org/t/udp-checksum-with-nftables/161522/11 the vm should have checksum offloading disabled * Routed: fix smoke test due to wrong cidrlist of egress rules and missing ingress rule from VR * PR 9346: fix lint error schema-41910to42000.sql * PR 9346: ui polish v1 * PR 9346: create VPC with cidrsize * Routed: fix test failures with test_network_ipv6 and test_vpc_ipv6 due to 'ssh -q' * Routed: fix /usr/local/cloud/systemvm/ are removed after SSVM/CPVM reboot * Routed: fix IP of additional nics of VPC VR is not gateway * PR 9346: fix cidrsize check when create VPC with cidrsize * Routed: fix test/integration/smoke/test_ipv4_routing.py:279:16: E713 test for membership should be 'not in' * PR9346: fix/Update api * PR 9346: set response object name * PR9346: UI refactor and small fixes * PR9346: change return type of getNetworkMode * PR9346: move IPv4 subnet to seperated tab * PR9346: revert IpRangesTabGuest.vue back to original * PR9346: fix remove ipv4 subnet on UI * PR9346: fix test_ipv4_routing.py * AS Number Range Management * Create AS Number Range for a Zone * Fix build * Add ListASNRange and fix create ASN range * Add List AS numbers * Add UI for AS Numbers * Fix UI and filter AS Numbers * Add AS Number on Isolated network creation and refactor UI and response * Release AS Number * Add network offering new columns * Add UI support to view and add AS number and configure network offering * Automatically assign AS Number if not specify AS number * update variable name * Fix routing mode check * UI: Only allow selecting AS number when routing mode is Dynamic and specifyAsNumber is true * UI: Only pass AS number when supported by the network offering * Release AS number on network deletion * Add deleteASNRange command (#81) * API: List ASNumbers by asnumber (#83) --------- Co-authored-by: Pearl Dsilva <pearl1594@gmail.com> * AS number management extensions * Support AS number on VPC tier creation based on the offering * Fix delete AS Range * Fix UI values * UI: Minor fix for releasing AS number * UI: Move management of AS Range to Zone details view * Fix specify_as_number column in network_offering table to set the default false * Add events for AS number operations * Allow users to list AS Numbers and fix network form for Normal users * Add AS number details to list networks response * Fix Allocated time format * Fix Allocated time format * support in details view too * Fix: Do not release AS number if acquired network requires AS number * Fix: Do not release AS number if acquired network requires AS number * Fix typo * Fix allocated release * Fix event type * UI: Add Routing mode and Specify AS to the network offering details * UI: Add Routing mode and Specify AS to the network offering details * Address comment * Fix release AS number of network deletion * Fix release AS number of network deletion * Fix * Restore release to its place based on the boolean * Rename boolean * API: Add networkId as listASNumber parameter * Add Network name to the search view filter for AS numbers * Present allocated time in human readable format - Pubilc IP / AS Numbers * Add account / domain filter for AS numbers * Add support for AS numbers on VPC offerings * Refactor AS number allocation to VPC and non VPC isolated networks * Checkstyle * Add support for AS numbers on VPC offerings * extend vpc offering view and vpcoffering response * merge https://github.com/shapeblue/cloudstack-playtika/pull/115 and change network_id of as_numbers to include vpc_id * Display AS number of VPC tiers as the AS number of the VPC * extend asnumber response and ui support * improve UI and as number response to view VPC details * List only dynamic offerings for vpc tiers with specify as numbers * Fix release AS number * Fix AS number displayed as 0 when no AS number assigned * Fix VPC offering creation without specify AS --------- Co-authored-by: nvazquez <nicovazquez90@gmail.com> * Fix release AS number on VPC deletion * Update server/src/main/java/com/cloud/dc/BGPServiceImpl.java * Update server/src/main/java/com/cloud/dc/BGPServiceImpl.java * Fix missing column on asnumber table * Fix listASNumbers API to support vpcid and obtain AS number from vpc for tiers * Prevent listing 0 AS number for VPC * Fix create Isolated Network form * Update server/src/main/java/com/cloud/network/vpc/VpcManagerImpl.java * Update server/src/main/java/com/cloud/network/vpc/VpcManagerImpl.java * Dynamic: move routingmode/specifyasn after networkmode in AddNetworkOffering.vue on UI * Dynamic: fix ip4routing in network response * Dynamic/systemvm: add FRR to systemvm template * Dynamic: BGP peers (DB,VO,Dao) * Dynamic: BGP peers (VR/server) * Dynamic: v3 - remove BgpPeer class - fix vpc vr has bgp peers of only 1 tier - rename ip4_cidr to guest_ip4_cidr - rename ip6_cidr to guest_ip6_cidr - generate /etc/frr/frr.conf - apply BGP peers on Dynamic-Routed network even if there is no BGP peers * Dynamic v4: fix vpc vr - fix duplicated guest cidr in frr.conf in vpc vr todo - restart frr / reload frr (reload will cause bgp session to Policy state) - apis for bgp peers - assign/release bgp peer from/to network * Dynamic v5: add apis for bgp peers * Dynamic v6: fix bugs - set response object name - remove required as number when update - fix checks when update - allow regular users to list bgp peers * Dynamic v7: move apis to bgp sub-dir * Dynamic v8: add tab for manage BGP peers on UI * Dynamic v9: fix update bgp with same config * Dynamiv v10: add changeBgpPeersForNetworkCmd * Dynamic v11: create network with bgppeerids - create network with bgppeerids - add marvin classes - add smoke tests - remove uuid from bgp_peer_network_map - fix created/removed in bgp_peer_network_map - remove bgppeers when remove a network - UI: fix delete bgp peer * Dynamic v12: add test for vpc tiers * Dynamic v13: bug fixes - fix change BGP peers for network in Allocated state - fix listing network returns removed record - fix all vpc tiers have the same settings - remove BGP peers as part of network removal - remove FRR settings for vpc tiers without any BGP peers - UI: fix no error msg when change BGP peers * Dynamic v14: assign BGP Peers for VPC instead of VPC tiers - create vpc with bgppeerids - do not allow create/update vpc tier with bgppeerids - apply all bgp peers when create/delete a vpc tier - UI: change bgp peers for vpc - test: update tests on vpc * Dynamic: fix build errors after merging as number PR * Dynamic: fix TODOs * Dynamic: fix smoke test on VPC * Allow creation of networks by users with as numbers * Address review comments * Move BGPService to bgp package and inject it on BaseCmd * Revert changes for CKS and address more comments * Display left side menu option for AS number only for root admin * Dynamic: create/update BGP peer with details refer to https://docs.frrouting.org/en/latest/bgp.html * Dynamic: fix build error and remove access to ListBgpPeers cmd for regular users * Dynamic: assign all zone BGP peers to user networks * Dynamic: show BGP peer info of networks only for root admin * AS number: disable specifyasnumber for non-NSX offerings * Dynamic: pass bgppeer details to command and fix typo with ip6 addr * Dynamic: list BGP peers by isdedicated, and fix change bgppeers for network/vpc * Dynamic: add UI labels * Dynamic: add bgp peers to vpc response * Dynamic: list bgp peers by keyword, fix list by asnumber * Dynamic: fix list bgppeers by keyword and db schema * Dynamic: fix list bgppeers do not return dedicated peers * Dynamic: update UI when create network/vpc offering * Update server/src/main/java/com/cloud/configuration/ConfigurationManagerImpl.java Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * Update tools/marvin/setup.py * Dynamic: network mode must be same when update a network with new offering * Dynamic: add method networkModel.isAnyServiceSupportedInNetwork * Dynamic: rename APIs and classes * Dynamic: fix unit tests due to previous changes * Dynamic: validateNetworkCidrSize when auto-create subnet * Dynamic: check AS number overlap * Dynamic: add ActionEvent * Dynamic: small code optimization * Dynamic: fix ui bugs after api rename * Dynamic: add marvin and test for ASN ranges and AS numbers * Dynamic: add account setting use.system.bgp.peers also - change the default value of routed.ipv4.vpc.max.cidr.size and routed.ipv4.vpc.min.cidr.size - change the category of settings * static: fix ui error when delete zone ipv4 subnets * static: small UI polish * Dynamic: throw exception when as number is required but not passed * Dynamic: fix typo when create FRR directory which causes network deletion failures * Dynamic: connect to ALL (or ALL dedicated) BGP peers if no BGP peer mapping for the network/vpc * Dynamic: throw exception when as number is required for VPC but not passed * Dynamic: list bgp peers by useSystemBgpPeers * Dynamic: fix frr config in VPC VR when change bgp peers * Dynamic: create frr config even if there is no VPC tiers * Dynamic: list bgp peers by zoneid (required for account) and account * Dynamic: only apply FRR config for vpc tiers with dynamic routing * Dynamic: donot send commands to router if commands size is 0 * Dynamic: fix 'new IPv6 address is not valid' when update bgp peer without IPv6 * Dynamic: throw exception if fail to allocate AS number when create network/vpc with dynamic routing * Dynamic: enable ipv6 unicast and 'ip nht resolve-via-default' * Dynamic: delete network/vpc if fail to allocate AS number when create network/vpc with dynamic routing * test: add unit tests for ASN APIs * test: add unit tests for core module * test: add unit tests for API responses * test: add unit tests for BgpPeerTO * test: add minor changes * test: add tests for create/delete/update/list RoutingFirewallRuleCmd * Static: show ip4 routes for vpc tiers * test: fix smoke test failure caused by type change of as number * test: add test for Ipv4SubnetForZoneCmd * test: add test for Ipv4SubnetForGuestNetworkCmd and BgpPeerCmd * UI: do not show redundant router when network mode is ROUTED as RVR is not supported * UI: hide 'Conserve mode' when networkmode is ROUTED * test: add unit tests for ListASNumbersCmdTest * Static: remove allocated IPv4 subnet when delete a network or vpc * test: add unit tests for BgpPeersRules * Dynamic: set ipv4routing from network offering * server: list as numbers and ipv4 subnets by keyword * server: remove dedicated bgp peers and ipv4 subnets when delete an account or domain * server: fix dedicated ipv4 subnet is allocated to other accounts * UI: fix allocated time format * server: ignore project is projectid is -1 so bgppeers/ipv4subnets works in project view * UI: add project column to bgp peers and ipv4 subnets * server: fix list AS numbers by domain admin or normal user * server: fix network creation when ipv4 subnet is dedicated * UI: polish network.js * Dynamic: fix frr config for ipv6 routing * Static routing: support cks cluster * Static: get/create IPv4 subnet from dedicated subnets at first * Dynamic: add BGP peers tab * Static: remove redundant loops * api: add since to api and response * server: add unit tests --------- Co-authored-by: Nicolas Vazquez <nicovazquez90@gmail.com> Co-authored-by: Pearl Dsilva <pearl1594@gmail.com> Co-authored-by: Harikrishna Patnala <harikrishna.patnala@gmail.com> Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com> Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com> |
|
Rohit Yadav
|
85765c3125
|
backup: simple NAS backup plugin for KVM (#9451)
This is a simple NAS backup plugin for KVM which may be later expanded for other hypervisors. This backup plugin aims to use shared NAS storage on KVM hosts such as NFS (or CephFS and others in future), which is used to backup fully cloned VMs for backup & restore operations. This may NOT be as efficient and performant as some of the other B&R providers, but maybe useful for some KVM environments who are okay to only have full-instance backups and limited functionality. Design & Implementation follows the `networker` B&R plugin, which is simply: - Implement B&R plugin interfaces - Use cmd-answer pattern to execute backup and restore operations on KVM host when VM is running (or needs to be restored) - instead of a B&R API client, relies on answers from KVM agent which executes the operations - Backups are full VM domain snapshots, copied to a VM-specific folders on a NAS target (NFS) along with a domain XML - Backup uses libvirt feature: https://libvirt.org/kbase/live_full_disk_backup.html orchestrated via virsh/bash script (nasbackup.sh) as the libvirt-java lacks the bindings - Supported instance volume storage for restore operations: NFS & local storage Refer the doc PR for feature limitations and usage details: https://github.com/apache/cloudstack-documentation/pull/429 Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com> Co-authored-by: Pearl Dsilva <pearl1594@gmail.com> Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com> Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com> |
|
Abhisar Sinha
|
605534b417
|
feature: Shared Storage Filesystem as a First Class Feature (#9208)
This PR implements Storage filesystem as a first class feature. https://cwiki.apache.org/confluence/display/CLOUDSTACK/Storage+Filesystem+as+a+First+Class+Feature Documentation PR: apache/cloudstack-documentation#420 Co-authored-by: Wei Zhou <weizhou@apache.org> |
|
John Bampton
|
c923e673cf
|
pre-commit: add `XML` files to the `trailing-whitespace` check (#9131) | |
Abhishek Kumar
|
3e6900ac1a
|
api,server: purge expunged resources (#8999)
This PR introduces the functionality of purging removed DB entries for CloudStack entities (currently only for VirtualMachine). There would be three mechanisms for purging removed resources:
Background task - CloudStack will run a background task which runs at a defined interval. Other parameters for this task can be controlled with new global settings.
API - New admin-only API purgeExpungedResources. It will allow passing the following parameters - resourcetype, batchsize, startdate, enddate. Currently, API is not supported in the UI.
Config for service offering. Service offerings can be created with purgeresources parameter which would allow purging resources immediately on expunge.
Following new global settings have been added:
expunged.resources.purge.enabled: Default: false. Whether to run a background task to purge the expunged resources
expunged.resources.purge.resources: Default: (empty). A comma-separated list of resource types that will be considered by the background task to purge the expunged resources. Currently only VirtualMachine is supported. An empty "value will result in considering all resource types for purging
expunged.resources.purge.interval: Default: 86400. Interval (in seconds) for the background task to purge the expunged resources
expunged.resources.purge.delay: Default: 300. Initial delay (in seconds) to start the background task to purge the expunged resources task.
expunged.resources.purge.batch.size: Default: 50. Batch size to be used during expunged resources purging.
expunged.resources.purge.start.time: Default: (empty). Start time to be used by the background task to purge the expunged resources. Use format yyyy-MM-dd or yyyy-MM-dd HH:mm:ss.
expunged.resources.purge.keep.past.days: Default: 30. The number of days in the past from the execution time of the background task to purge the expunged resources for which the expunged resources must not be purged. To enable purging expunged resource till the execution of the background task, set the value to zero.
expunged.resource.purge.job.delay: Default: 180. Delay (in seconds) to execute the purging of an expunged resource initiated by the configuration in the offering. Minimum value should be 180 seconds and if a lower value is set then the minimum value will be used.
Documentation PR: apache/cloudstack-documentation#397
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Wei Zhou <weizhou@apache.org>
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
|
|
|
Wei Zhou
|
aab1361949
|
server: fix mgmt server 503 error by renaming kubernetesClusterHelper to kubernetesServiceHelper | |
Daan Hoogland
|
cb9b3134f7 | Merge branch '4.19' | |
|
Abhishek Kumar
|
ce9b2c52f3
|
cks: fix events (#9070)
Fixes #8043 Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> |
|
|
Abhishek Kumar
|
be552fdce9
|
feature: webhooks (#8674)
* api,server,ui: weebhoks feature Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * changes Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * registry of message busses * test bus Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * refactor Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * test Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix and refactor Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * changes for webhook dispatch history Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * changes, initial ui Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * improvements Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * changes for account webhook cleanup Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix remaining event bus usage Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * changes for testing webhook dispatch Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * wip Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix test Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * make element Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * missing Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * buid fix * fix lint Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * changes for project delete check Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * add collapse in create Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * ui fix and refactor for eventditributor publish Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * update org.json and add json validation Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * schema fixes Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * wordings Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * ui: improve progress button Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * ui improvements Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * remove unrelated change Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * search and count Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * add payloadurl in info Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * positive progress Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix hmac key Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * create webhook form fixes Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * refactor, address feedback Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * indentation Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix filters Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * remove test eventbus Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * default scope be Local Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * add lifecycle smoke test Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * add test for webhook deliveries Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * refactor Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix lint Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * refactor - losgs and others Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * unit tests Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix lint Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * build fix Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * smoke test fix, log refactor Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * get bean from all components Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * ui: missing label Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * address review comments Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * add some more tests Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * lint Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * rename setting Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * upgrade: move 4.19.0->4.20.0 to 4.19.1->4.20.0 * fix test delivery layout Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix webhook secret display Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * add http to payloadurl when no scheme Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * allow removing secretkey for webhook Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix update sslverification Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * disallow same payload url for same account Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix delivery with url w/o scheme Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * api: listApis should return params based on caller Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * wip changes Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * Update engine/schema/src/main/resources/META-INF/db/schema-41900to42000.sql * remove unique constraint for now Constraint is present in Java code validations Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fixes Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * ui: add option to delete multiple deliveries Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * add filter for deliveries, delete api start/endtime support Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * do not throw error when no deliveries removed Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * ui: fix deliveries table column sorting, time filter cancel Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * remove isDebugEnabled wrapping * merge fix Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> --------- Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> Co-authored-by: Daan Hoogland <daan@onecht.net> Co-authored-by: Wei Zhou <weizhou@apache.org> |
|
|
Daan Hoogland
|
a358c9a410 |
Merge release branch 4.19 to main
* 4.19: New feature: Import/Unamange DATA volume from storage pool (#8808) |
|
|
Wei Zhou
|
0b857def68
|
New feature: Import/Unamange DATA volume from storage pool (#8808) | |
Pearl Dsilva
|
6dc3d06037
|
NSX integration (#7919)
* NSX integration - skeletal code * Fix module not loading on startup * add upgrade path and daos \n add nsx controller command * add support for adding and listing nsx provider to a zone * add license * add default VPC offering and update upgrade path * add global setting to enable nsx plugin * add delete nsx controller operation * add nsxresource * add NSX resource , api client, create tier1 gw * update db * update response and add license * Add support to create and delete nsx tier-1 gateway * add license * cleanup and add skeletal code for network creation * add create/delete segment and UI integration * add license * address code smells - part 1 * fix test / build failure * NSX integration - skeletal code * Fix module not loading on startup * add upgrade path and daos \n add nsx controller command * add support for adding and listing nsx provider to a zone * add license * add default VPC offering and update upgrade path * add global setting to enable nsx plugin * add delete nsx controller operation * add nsxresource * add NSX resource , api client, create tier1 gw * update db * update response and add license * Add support to create and delete nsx tier-1 gateway * add license * cleanup and add skeletal code for network creation * add create/delete segment and UI integration * add license * address code smells - part 1 * fix test / build failure * add ui changes + update nsx_provider table transport zones + use NSX broadcast domain for add nics to router * ui: fix password field, and backend changes * add route advertisement * update offering * update offering * add sleep before deletion of vpc / tier g/w for ports to be removed * move creation of segments to design phase * change provider to VPC router for Dhcp & dns service in an nsx offering * Add public nic for NSX * reserve first IP (after g/w) of subnet for router nic - NSX * revert reserving 1st IP in vpc segments * [NSX] Create a DHCP relay and add it to a VPC tier segment (#107) * Create DHCP relay command and execute request * In progress integrate with networking * Create DHCP relay config on the network VR allocation * Revert domain router dao changes * Create DHCP relay con VR nic plug to NSX network * Link DHCP relay config to segment after creation * [NSX] Cleanup DHCP Relay config on segment deletion (#108) * Cleanup DHCP Relay config on segment deletion * update segment & relay name generators and call delete dhcprelay after deletion of segment * address comment * [NSX] Fix DHCP relay config deletion was missing zone name (#8068) * [NSX] Refactor API wrapper operations (#8059) * [NSX] Refactor API wrapper operations * Big refactor * Address review comment * change network cidr to cidr to prevent NPE * add domain and zone names to the various networks - vpc & tier --------- Co-authored-by: Pearl Dsilva <pearl1594@gmail.com> * Nsx unit tests (#8090) * Add tests * add test for NsxGuestNetworkGuru * add unit tests for NsxResource * add unti tests for NsxElement * cleanup * [NSX] Refactor API wrapper operations * update tests * update tests - add nsxProviderServiceImpl test * add unit test - NsxServiceImpl * add license * Big refactor * Address review comment * change network cidr to cidr to prevent NPE * add domain and zone names to the various networks - vpc & tier * fix tests --------- Co-authored-by: nvazquez <nicovazquez90@gmail.com> * modify NSX resource naming convention (#8095) * modify NSX resource naming convention * remove unused imports * add a setup phase between desgin and implementation of a network for intermediary steps * add method to all classes * NSX: Refactor Network & VPC offering (#8110) * [NSX] Refactor API wrapper operations * Network offering changes for NSX * fix services and provider combination * address comments: rename param * update nsx_mode parameter --------- Co-authored-by: nvazquez <nicovazquez90@gmail.com> * fix test * [NSX] Allow NSX isolated networks (#8132) * Add network offerings for NSX on isolated networks * Fix offerings creation * In progress NSX isolated network * Fixes * Fix NIC allocation to router * NSX: Add Step for Adding Public traffic network for NSX During zone creation (#8126) * NSX: Add Step for Adding Public traffic network for NSX * address comments and cleanup * address comment * remove indent * NSX: Create and Delete static NAT & Port forward rules (#8131) * NSX: Create and delete NSX Static Nat rules * fix issues with static nat * add static nat * Support to add and delete Port forward rules * add license * fix adding multiple pf rules * cleanup * fix lint check * fix smoke tests * fix smoke tests * Nsx add lb rule (#8161) * NSX: Create and delete NSX Static Nat rules * fix issues with static nat * add static nat * Support to add and delete Port forward rules * add license * fix adding multiple pf rules * cleanup * NSX: Add support to create and delete Load balancer rules * fix deletion of lb rules * add header file and update protocol detail * build failure fix * [NSX] Add SNAT support (#8100) * In progress add source NAT * Fix after merge * Fix tests * Fix NPE on isolated network deletion * Reserve source NAT IP when its not passed for NSX VPC * Create source NAT rule on VR NIC allocation * Fix update VPC and remove VPC to update and remove SNAT rule * Fix packaging * Address review comment * Fix build * fix build - unused import * Add defensive checks * Add missing design to NSX public guru --------- Co-authored-by: Pearl Dsilva <pearl1594@gmail.com> * NSX: Fix VR public NIC allocation (#8166) * NSX: fix LB member addition and deletion and add defensive checks (#8167) * Fix public NIC NPE on broadcast URI * NSX: Router Public nic to get IP from systemVM Ip range (#8172) * NSX: Router Public nic to get IP from systemVM Ip range * Fix VR IP address and setSourceNatIp command * NSX: hide systemVM reserved IP range SourceNAT * fix test --------- Co-authored-by: nvazquez <nicovazquez90@gmail.com> * fix test failure * test failure fix * [NSX] Fix update source NAT IP (#8176) * [NSX] Fix update source NAT IP * Fix startup * Fix API result * NSX - add LB route Advertizement (#8192) * [NSX] Add ACL types support (#8224) * NSX: Create segment group on segment creation * Add unit tests * Remove group for segment before removing segment * Create Distributed Firewall rules * Remove distributed firewall policy on segment deletion * Fix policy rule ID and add more unit tests * Fix DROP action rules and transform tests * Add new ACL rules * Fixes * associate security policies with groups and not to DFW and add deletion of rules * Fix name convention --------- Co-authored-by: Pearl Dsilva <pearl1594@gmail.com> * NSX: Fix creation of VPCs (#8320) * Fix ACL rules creation (#8323) * [NSX] Fix database views (#8325) * NSX: Add CKS Support & Firewall rules for Isolated Networks (#8189) * NSX: Add ALL LB IP to the list of route advertisements in tier1 * NSX: Support Source NAT on NSX Isolated networks * NSX: Cks Support * NSX: Create segment group on segment creation * Add unit tests * Remove group for segment before removing segment * Create Distributed Firewall rules * Remove distributed firewall policy on segment deletion * Fix policy rule ID and add more unit tests * Add support for routed NSX Isolated networks \n and non RFC 1918 compliant IPs * Add support for routed NSX Isolated networks \n and non RFC 1918 compliant IPs * Add Firewall rules * build failure - fix unit test * fix npes * Add support to delete firewall rules * update nsx cks offering * add license * update order of ports in PF & FW rules * fix filter for getting transport zones * CKS support changed - MTU updated, etc * add LB for CKS on VPC * address comments * adapt upstream cks logic for vpc * rever mtu hack * update UI changes as per upstream fix * change display test for CKS n/w offerings for isolated and VPC tiers * add extra line for linter * address comment * revert list change --------- Co-authored-by: nvazquez <nicovazquez90@gmail.com> * fix ui build failure * [NSX] Address SonarCloud Bugs (#8341) * [NSX] Address SonarCloud Bugs * Fix NSX API connection issues * NSX: Add unit tests to increase coverage (#8355) * NSX: Add unit tests * cleanup unused imports * add more unit tests * add tests for publicnsxnetworkguru * add license * fix build failures * address sonar comment * fix security hotspots * NSX: Add more unit tests (#8381) * NSX : Unit tests * remove unused imports * remove unused import causing build failure * fix build failures due to unused imports * fix build failure * fix test assertion * remove unused imports * remove unused import * Nsx UI zone bug (#8398) * NSX: Attempt to fix NSX Zone creation bug for public networks * fix zone wizard public traffic issue * add proper filtering of offerings based on VPC nsx mode * clean up console logs * NSX: Fix code smells and reported bugs (#8409) * NSX: Fix code smells and reported bugs * fox override issue * remove unused imports * fix test * refactor code to reduce complexity * add lisence * cleanup * fix build failure * fix build failure * address comments * test - add config to ignore certain files from test coverage * test exclusion of classes from test cov * rever pom changes * [NSX] Add more unit tests (#8431) * [NSX] Add more unit tests * More tests * Fix build errors * NSX: Prevent creation of L2 and Shared networks for NSX (#8463) * NSX: Prevent creation of L2 and Shared networks for NSX * add checks to backend to prevent creation of l2 and shared networks in nsx zones and filter only nsx offerings when creating isolated networks * cleanup * NSX: Fix code smells (#8436) * NSX: Fix code smells * Add changes to service creation logic * CKS: Add action to during firewall rule creation (#8498) * NSX,UI: Deduplicate network list when creating kubernetes clusters (#8513) * NSX: Make LB service selectable in network offering (#8512) * NSX: Make LB service selectable in network offering * fix label * address comments * address comments * NSX: Add appropriate error message when icmp type is set to -1 for NSX (#8504) * NSX: Add appropriate error message when icmp type is set to -1 for NSX * address comments * update text * fix test * fix test - build failure * fix test - build failure * NSX: Cleanup NSX resources during k8s cluster cleanup (#8528) * fix test failure * NSX: Improve segment deletion process (#8538) * NSX: Add passive monitor for NSX LB to test whether a server is available (#8533) * NSX: Add passive monitor for NSX LB to test whether a server is available * Add active monitors too * fix build failure * NSX: Add check for ICMP code / type for NSX zones (#8542) * NSX: Fix Routed Mode for Isolated and VPC networks (#8534) * NSX: Fix Routed Mode for Isolated and VPC networks * NSX: Fix Routed mode - add checks for ports added for FW rules * clean up code * fix build failure * NSX: Add retry logic with sleep to delete segments (#8554) * NSX: Add retry logic with sleep to delete segments * add logs * NSX: Fix custom ACL check (#2) * NSX: Fix custom ACL check * NSX: Fix custom ACL check * Nsx vpc routed mode (#5) * NSX: Fix VPC routed mode * NSX: VPC route mode * remove unnecessary changes * Nsx: Support internal LB (#4) * NSX: Support internal LB service in NSX * add lb removal logic * Fix UI issue hiding internal LB tab * Refactor method name --------- Co-authored-by: nvazquez <nicovazquez90@gmail.com> * NSX: Improve NSX resource cleanup process (#3) * Fix unit test * NSX: Add SourceNAT service to the default Routed offering for VPC (#13) * Fix VPC restart with cleanup (#12) * NSX: Fix ACL rule removal on replacement and fix rule order (#11) * NSX: fix smoke test failure for ACLs (#9) * Fix unit tests * Fix NSX plugin pom XML * NSX: Add support to re-order ACL rules (NSX FW rules) (#14) * [WIP] NSX: Add support to re-order ACL rules (NSX FW rules) * fix reordering of acl rules on all networks that it is associated to * clean up and attempt test fix * Fix tests * Remove unused import * tweak reorder logic --------- Co-authored-by: nvazquez <nicovazquez90@gmail.com> * Fix zone creation issue for internal load balancer * Fix * Fix unit test * fix logger * fix logger * fix logger * NSX: Fix VPC form to ignore source NAT IP when creating VPCs and fix label * Move SQL changes to the newest schema file * NSX: Last Fixes * Fix build --------- Co-authored-by: nvazquez <nicovazquez90@gmail.com> |
|
kishankavala
|
5651eab49c
|
ObjectStore Framework with MinIO and Simulator plugins (#7752)
This PR adds Object Storage feature to CloudStack. FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/%5BDRAFT%5D+CloudStack+Object+Store |
|
|
Vishesh
|
5362bad442
|
Storage Management (#7949) | |
|
Wei Zhou
|
bd52fa8a12
|
New feature: VNF templates and appliances integration (#8022) | |
|
Vishesh
|
ea90848429
|
Feature: Add support for DRS in a Cluster (#7723)
This pull request (PR) implements a Distributed Resource Scheduler (DRS) for a CloudStack cluster. The primary objective of this feature is to enable automatic resource optimization and workload balancing within the cluster by live migrating the VMs as per configuration.
Administrators can also execute DRS manually for a cluster, using the UI or the API.
Adds support for two algorithms - condensed & balanced. Algorithms are pluggable allowing ACS Administrators to have customized control over scheduling.
Implementation
There are three top level components:
Scheduler
A timer task which:
Generate DRS plan for clusters
Process DRS plan
Remove old DRS plan records
DRS Execution
We go through each VM in the cluster and use the specified algorithm to check if DRS is required and to calculate cost, benefit & improvement of migrating that VM to another host in the cluster. On the basis of cost, benefit & improvement, the best migration is selected for the current iteration and the VM is migrated. The maximum number of iterations (live migrations) possible on the cluster is defined by drs.iterations which is defined as a percentage (as a value between 0 and 1) of total number of workloads.
Algorithm
Every algorithms implements two methods:
needsDrs - to check if drs is required for cluster
getMetrics - to calculate cost, benefit & improvement of a migrating a VM to another host.
Algorithms
Condensed - Packs all the VMs on minimum number of hosts in the cluster.
Balanced - Distributes the VMs evenly across hosts in the cluster.
Algorithms use drs.level to decide the amount of imbalance to allow in the cluster.
APIs Added
listClusterDrsPlan
id - ID of the DRS plan to list
clusterid - to list plans for a cluster id
generateClusterDrsPlan
id - cluster id
iterations - The maximum number of iterations in a DRS job defined as a percentage (as a value between 0 and 1) of total number of workloads. Defaults to value of cluster's drs.iterations setting.
executeClusterDrsPlan
id - ID of the cluster for which DRS plan is to be executed.
migrateto - This parameter specifies the mapping between a vm and a host to migrate that VM. Format of this parameter: migrateto[vm-index].vm=<uuid>&migrateto[vm-index].host=<uuid>.
Config Keys Added
ClusterDrsPlanExpireInterval
Key drs.plan.expire.interval
Scope Global
Default Value 30 days
Description The interval in days after which old DRS records will be cleaned up.
ClusterDrsEnabled
Key drs.automatic.enable
Scope Cluster
Default Value false
Description Enable/disable automatic DRS on a cluster.
ClusterDrsInterval
Key drs.automatic.interval
Scope Cluster
Default Value 60 minutes
Description The interval in minutes after which a periodic background thread will schedule DRS for a cluster.
ClusterDrsIterations
Key drs.max.migrations
Scope Cluster
Default Value 50
Description Maximum number of live migrations in a DRS execution.
ClusterDrsAlgorithm
Key drs.algorithm
Scope Cluster
Default Value condensed
Description DRS algorithm to execute on the cluster. This PR implements two algorithms - balanced & condensed.
ClusterDrsLevel
Key drs.imbalance
Scope Cluster
Default Value 0.5
Description Percentage (as a value between 0.0 and 1.0) of imbalance allowed in the cluster. 1.0 means no imbalance
is allowed and 0.0 means imbalance is allowed.
ClusterDrsMetric
Key drs.imbalance.metric
Scope Cluster
Default Value memory
Description The cluster imbalance metric to use when checking the drs.imbalance.threshold. Possible values are memory and cpu.
|
|
Nicolas Vazquez
|
8b5ba13b81
|
plugins: Add Custom hypervisor minimal changes (#7692)
### Description Design document: https://cwiki.apache.org/confluence/display/CLOUDSTACK/%5BDRAFT%5D+Minimal+changes+to+allow+new+dynamic+hypervisor+type%3A+Custom+Hypervisor This PR introduces the minimal changes to add a new hypervisor type (internally named Custom in the codebase, and configurable display name), allowing to write an external hypervisor plugin as a Custom Hypervisor to CloudStack The custom hypervisor name is set by the setting: 'hypervisor.custom.display.name'. The new hypervisor type does not affect the behaviour of any CloudStack operation, it simply introduces a new hypervisor type into the system. CloudStack does not have any means to dynamically add new hypervisor types. The hypervisor types are internally preset by an enum defined within the CloudStack codebase and unless a new version supports a new hypervisor it is not possible to add a host of a hypervisor that is not in part of the enum. It is possible to implement minimal changes in CloudStack to support a new hypervisor plugin that may be developed privately This PR is an initial work on allowing new dynamic hypervisor types (adds a new element to the HypervisorType enum, but allows variable display name for the hypervisor) ##### Proposed Future work: Replace the HypervisorType from a fixed enum to an extensible registry mechanism, registered from the hypervisor plugin #### Feature Specifications - The new hypervisor type is internally named 'Custom' to the CloudStack services (management server and agent services, database records). - A new global setting ‘hypervisor.custom.display.name’ allows administrators to set the display name of the hypervisor type. The display name will be shown in the CloudStack UI and API. - In case the ‘hypervisor.list’ setting contains the display name of the new hypervisor type, the setting value is automatically updated after the ‘hypervisor.custom.display.name’ setting is updated. - The new Custom hypervisor type supports: - Direct downloads (the ability to download templates into primary storage from the hypervisor hosts without using secondary storage) - Local storage (use hypervisor hosts local storage as primary storage) - Template format: RAW format (the templates to be registered on the new hypervisor type must be in RAW format) - The UI is also extended to display the new hypervisor type and the supported features listed above. - The above are the minimal changes for CloudStack to support the new hypervisor type, which can be tested by integrating the plugin codebase with this feature. #### Use cases This PR allows the cloud administrators to test custom hypervisor plugins implementations in CloudStack and easily integrate it into CloudStack as a new hypervisor type ("Custom"), reducing the implementation to only the hypervisor supported specific storage/networking and the hypervisor resource to communicate with the management server. - CloudStack admin should be able to create a zone for the new custom hypervisor and add clusters, hosts into the zone with normal operations - CloudStack users should be able to execute normal VMs/volumes/network/storage operations on VMs/volumes running on the custom hypervisor hosts |
|
|
John Bampton
|
6f4503488b
|
pre-commit: apply `end-of-file-fixer` to all files (#7551) | |
|
Vishesh
|
fa3f2a75eb
|
feature: VM Scheduler (#7397)
This PR adds a feature to setup schedules to stop/start/restart your VMs. Documentation PR: apache/cloudstack-documentation#313 Related issue: #3387 |
|
David Jumani
|
941cc83372
|
Feature: Safely shutdown cloudstack (#6755)
Co-authored-by: dahn <daan.hoogland@gmail.com> |
|
Harikrishna
|
a3feccf70c
|
User two factor authentication (#6924)
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com> |
|
|
Nicolas Vazquez
|
b2fbe7bb12
|
console: Console access enhancements (#6577)
This PR creates a new API createConsoleAccess to create VM console URL allowing it to connect using other UI implementations. To avoid reply attacks, the console access is enhanced to use a one time token per session New configuration added: consoleproxy.extra.security.validation.enabled: Enable/disable extra security validation for console proxy using a token Documentation PR: apache/cloudstack-documentation#284 |
|
João Jandre
|
117ce1aac4
|
Create password policies configurations (#6567)
Co-authored-by: João Paraquetti <joao@scclouds.com.br> |
|
Abhishek Kumar
|
4a914aa88d
|
network: ipv6 static routes (#5786)
* wip Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * IPv6: configure VR of isolated networks * IPv6: add default IPv6 route in VR of isolated networks * Reformat server/src/main/java/com/cloud/network/NetworkServiceImpl.java * IPv6: update network to offering which support IPv6 * IPv6: update vm nic ipv6 address when update network to new offering * IPv6: configure VPC VR to support multiple tiers with IPv6 * IPv6: add RDNSS in radvd.conf * IPv6/UI: support ipv6 protocols in Network ACL * wip Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * changes for diagnostics Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * more import fromo #5594 Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * IPv6: fix wrong public ipv6 in VPC VR * changes Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * Update server/src/main/java/com/cloud/configuration/ConfigurationManagerImpl.java Co-authored-by: dahn <daan.hoogland@gmail.com> * ui: fix add ipv6 prefix labels, message Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * ui: label fix Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * logging fix Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * test fix Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * changes Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * minor ui refactor Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * ip6 events Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * ip6 usage Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * unused Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * slaac based public ip Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * remove unused Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * diagnostics fix for vr Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * firewall changes Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * alert and show ipv6 usage Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * change for network response Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * ipv6 network test Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * changes Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * ui: fix ipaddress listing Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * wip Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix simulator Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * changes Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * changes Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * test fix Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * test and fixes Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * test temp change revert Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fixes Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * use uuid Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * event syntax fix Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * wip Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * review comments Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * assign vlan public IP for dualstack only if both protocols present on same vlan Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * internetprotocol in networkofferingresponse Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * add tcp, udp Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * support vpc with ipv6 only on same vlan - adds new internet protocol param to createVpcOffering API - When DualStack internet protocol is selected for the VPC offering, tiers with network with or without IPv6 support can be deployed. - When IPv4 internet protocol is used for the VPC offering, tiers with network with only IPv4 support can be deployed Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * change and fix allow VPC with IPv4 protocol to deploy tiers with IPv6 Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * test fix Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * ui fixes Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix multiple routes, network guest ipv6 gateway Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * address review comments Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * stop radvd on backup VR Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix router redundant status with ipv6 Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * disable radvd for backup vr Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * correctly set ipv6 in redundant router case Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * remove unused code Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix connection Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * ui: don't show all protocol for egress Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix guest ipv6 for redundant VRs Redundant VRs will not be assigned an IPv6 by ACS and guest netwrok gateway will be added as IPv6 for guest interface by systemvm scripts during setting redundant state of the VR. Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix missing ipv6 on redundant vr Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix syntax Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * ui: fix vpc tier redirect to show details When redirecting to VPC tier, details tab should be active by default Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * restart radvd on primary redundant vr Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * check for ipv6 values Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * remove old ui change Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix condition Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * remove gateway from backup vr Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * network upgrade fail early when IPv6 network cannot be allocated fail before shutting down the network Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix radvd not running on RVR Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * prepare radvd.conf once Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix job polling Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix RVR for vpc with ipv6 Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix ipv6 network acls Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * Update CsConfig.py * add check Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * test: vpc offering test Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * test: add negative tests for guest prefix, public range Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * add default ipv6 route for primary Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix dadfailed on vpc rvr Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * ui: fix add iprange form, dedicate action visibility Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix adding, deleting ipv6 range Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix failing test Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix missing destination cidr in ipv6 firewall Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix ipv6 nftables rules Allow storing linger IPv6 CIDRs in DB Specify all port range for TC{, UDP protocol rules withot ports Fix adding nft rules by creating chains first Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix icmpv6 type, code Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix icmp type, code Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * test: add more for ipv6 network Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * add warning message for egress policy in ipv6 fw rule Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * ui,server: update ipv6 vlan range Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * subnet operations inside transaction Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * server: persistent public IPv6 for network Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * ui: fix action alignment Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix vpc acl for tiers Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix removing network placeholder nic Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix acl rules for ip version Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix placeholder nic and nd-neighbor block issue Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * test for redundant nw Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix ping Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * systemvm: uprgade to debian 11.3.0 * ipv6: enable ipv6 in sysctl config in bootstrap.sh * VR: fix KeyError: 'nic_ip6_cidr' * build fix for latest event changes Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com> Co-authored-by: Wei Zhou <weizhou@apache.org> Co-authored-by: dahn <daan.hoogland@gmail.com> |
|
Daniel Augusto Veronezi Salvador
|
39fad2d9d7
|
KVM disk-only based snapshot of volumes instead of taking VM's full snapshot and extracting disks (#5297)
* Refactor create volume snapshot with running VM * Refactor create volume snapshot with stopped VM * Refactor create volume from snapshot * Refactor create template from snapshot * Refactor volume migration (migrateVolume/ migrateVirtualMachineWithVolume) * Refactor snapshot deletion * Refactor snapshot revertion * Adjusts and fix cherry-pick conflicts * Remove diffuse tests * Add validation to add flag '--delete' on command 'virsh blockcommand' only if libvirt version is equal or higher 6.0.0 * Expunge temporary snapshot only if template creation is from snapshot * Extract strings to constant * Remove unused imports * Fix error on revert backed up snapshot * Turn method's return to void as it is not used * Rename method in SnapshotHelper * Fix folder creation when using SharedMountPoint pool * Remove static import * Remove unnused method * Cover take snapshot in centos 7 * Handle right snapshot flag according to qemu version Co-authored-by: GutoVeronezi <daniel@scclouds.com.br> |
|
|
Wei Zhou
|
88d77c86a7
|
server: fix NPE when router.service.offering is set due to service/disk offering refactoring (#6120)
* server: move files in server/src/main/java/org/cloud/network/router/deployment/ * server: fix NPE when router.service.offering is set due to service/disk offering refactoring |
|
dahn
|
a6d9fa61b9
|
Role escalation prevention (#5879)
* prevent role access escallation * hierarchy issue fixed * create api list in account manager for checking new account access * full api list check * strange role restriction removed for BareMetal * add role check on upfdate account as well * more selective use of api checkers * error msg and var name Co-authored-by: Daan Hoogland <dahn@onecht.net> |
|
|
Pearl Dsilva
|
3f827ef22b
|
UI: Support to upload resource icons (#5157)
* Resource Icon support - backend * Add API support for resourceicon * update reponse params + ui support * Add exclusive list api for icons and UI changes * refactor upload view * UI changes to support resource icon wherever necessary * convert api to POST + refactor icon view * Add response name to list API + cosmetic changes in UI * Added support for the following: resource icon support for vpcs, networks, domains, and projects add icons to list view if reosurces support icons to be added support for showing project icons in the project switching drop-down menu * List resourceicon cmds to be allowed for user role too Users to inherit account icon if present (in listUsers response) Move common code to plugin.js Add icon to project list view - while switching between projects - Dashboard page Show icons against zones - Capacity Dashboard view Show user / account icon at the login button if present * cosmetic changes * optimize ui code * fix reload issue for domain view * add access check for delete operation * ui-related changes to show iso icons * iso image in uservm response * add icons to custom form's list resources * some more custom forms aligned to show icon for resources * conmitic changes + add listing of icons to listdomainchildren cmd * Add backend/server-side validation for base64 string passed for image * change preview border * preselect zone if there's only one * add default icon * show icon for network list in deploy vm view * add custom icons if any to the import-export VM view * preselect zone persistence on clearing cache * prevent root vol from inheriting template/iso icon * show tempalte icon in the info card details * fix icon not being show on hard-refresh / initial traversal * fx success message |
|
|
Nicolas Vazquez
|
413d10dd81
|
server: Extend the Annotations framework (#5103)
* Extend addAnnotation and listAnnotations APIs * Allow users to add, list and remove comments * Add adminsonly UI and allow admins or owners to remove comments * New annotations tab * In progress: new comments section * Address review comments * Fix * Fix annotationfilter and comments section * Add keyword and delete action * Fix and rename annotations tab * Update annotation visibility API and update comments table accordingly * Allow users seeing all the comments for their owned resources * Extend comments for volumes and snapshots * Extend comments to multiple entities * Add uuid to ssh keypairs * SSH keypair UI refactor * Extend comments to the infrastructure entities * Add missing entities * Fix upgrade version for ssh keypairs * Fix typo on DB upgrade schema * Fix annotations table columns when there is no data * Extend the list view of items showing they if they have comments * Remove extra test * Add annotation permissions * Address review comments * Extend marvin tests for annotations * updating ui stuff * addition to toggle visibility * Fix pagination on comments section * Extend to kubernetes clusters * Fixes after last review * Change default value for adminsonly column * Remove the required field for the annotationfilter parameter * Small fixes on visibility and other fixes * Cleanup to reduce files changed * Rollback extra line * Address review comments * Fix cleanup error on smoke test * Fix sending incorrect parameter to checkPermissions method * Add check domain access for the calling account for domain networks * Fix only display annotations icon if there are comments the user can see * Simply change the Save button label to Submit * Change order of the Tools menu to provent users getting 404 error on clicking the text instead of expanding * Remove comments when removing entities * Address review comments on marvin tests * Allow users to list annotations for an entity ID * Allow users to see all comments for allowed entities * Fix search filters * Remove username from search filter * Add pagination to the annotations tab * Display username for user comments * Fix add permissions for domain and resource admins * Fix for domain admins * Trivial but important UI fix * Replace pagination for annotations tab * Add confirmation for delete comment * Lint warnings * Fix reduced list as domain admin * Fix display remove comment button for non admins * Improve display remove action button * Remove unused parameter on groupShow * Include a clock icon to the all comments filter except for root admin * Move cleanup SQL to the correct file after rebasing main Co-authored-by: davidjumani <dj.davidjumani1994@gmail.com> |
|
Suresh Kumar Anaparti
|
958182481e |
cloudstack: make code more inclusive
Inclusivity changes for CloudStack - Change default git branch name from 'master' to 'main' (post renaming/changing default git branch to 'main' in git repo) - Rename some offensive words/terms as appropriate for inclusiveness. This PR updates the default git branch to 'main', as part of #4887. Signed-off-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com> Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com> |
|
|
Pearl Dsilva
|
b464fe41c6
|
server: Secondary Storage Usage Improvements (#4053)
This feature enables the following: Balanced migration of data objects from source Image store to destination Image store(s) Complete migration of data setting an image store to read-only viewing download progress of templates across all data stores Related Primate PR: apache/cloudstack-primate#326 |
|
|
Pearl Dsilva
|
c578004fe5
|
projects: Role based users in Projects (#4128)
Enabling Role Based users in projects Primate PR related to the FR: apache/cloudstack-primate#382 Doc PR: https://github.com/apache/cloudstack-documentation/pull/145 Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com> Co-authored-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com> |
|
|
Nicolas Vazquez
|
8c1d749360
|
[VMware] Enable unmanaging guest VMs (#4103)
* Enable unmanaging guest VMs * Minor fixes * Fix stop usage event only if VM is not stopped when unmanaging * Rename unmanaged VMs manager * Generate netofferingremove usage event if VM is not stopped * Generate usage event VM snapshot primary off when unmanaging |
|
|
Nicolas Vazquez
|
efe00aa7e0
|
[KVM] Rolling maintenance (#3610) | |
|
Rohit Yadav
|
318924d801
|
CloudStack Backup & Recovery Framework (#3553) | |
|
Abhishek Kumar
|
0f5b0e67f8
|
VM ingestion (#3606)
The VM ingestion feature allows CloudStack to discover, on-board, import existing VMs in an infra. The feature currently works only for VMware, with a hypervisor agnostic framework which may be extended for KVM and XenServer in future. |
|
Paul Angus
|
be97470d83 |
Get Diagnostics: Download logs and diagnostics data from SSVM, CPVM, Router (#3350)
* * Complete API implementation * Complete UI integration * Complete marvin test * Complete Secondary storage GC background task * improve UI labels * slight reword and add another missing description * improve download message clarity * Address comments * multiple fixes and cleanups Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com> * fix more bugs, let it return ip rule list in another log file Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com> * fix missing iprule bug Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com> * add support for ARCHIVE type of object to be linked/setup on secstorage Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com> * Fix retrieving files for Xenserver * Update get_diagnostics_files.py * Fix bug where executable scripts weren't handled * Fixed error on script cmd generation * Do not filter name for log files as it would override similar prefix script names * Addressed code review comments * log error instead of printstacktrace * Treat script as executable and shell script * Check missing script name case and write to output instead of catching exception * Use shell = true instead of shlex to support any executable * fix xenserver bug * don't set dir permission for vmware * Code review comments - refactoring * Add check for possible NPE * Remove unused imoprt after rebase * Add better description for configs Co-authored-by: Nicolas Vazquez <nicovazquez90@gmail.com> Co-authored-by: Rohit Yadav <rohit@apache.org> Co-authored-by: Anurag Awasthi <anurag.awasthi@shapeblue.com> |
|
|
Nicolas Vazquez
|
a75444a585
|
KVM: DPDK live migrations (#3365)
* DPDK live migrations * Remove DPDK created ports if VM migration fails or prepare migration fails * Rename DPDK classes lowercase |
|
|
Nicolas Vazquez
|
501aa7cd91
|
DPDK vHost User mode selection (#3153)
* DPDK vHost User mode selection * SQL text field and DPDK classes refactor * Fix NullPointerException after refactor * Fix unit test * Refactor details type |
|
|
Rohit Yadav
|
3424d9e6ff |
Merge remote-tracking branch 'origin/4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com> |
|
Dingane Hlaluku
|
40af32b1b9 |
diagnostics: new diagnostics admin API for system VMs (#2721)
This is a new feature for CS that allows Admin users improved troubleshooting of network issues in CloudStack hosted networks. Description: For troubleshooting purposes, CloudStack administrators may wish to execute network utility commands remotely on system VMs, or request system VMs to ping/traceroute/arping to specific addresses over specific interfaces. An API command to provide such functionalities is being developed without altering any existing APIs. The targeted system VMs for this feature are the Virtual Router (VR), Secondary Storage VM (SSVM) and the Console Proxy VM (CPVM). FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+Remote+Diagnostics+API ML discussion: https://markmail.org/message/xt7owmb2c6iw7tva |
|
|
Rohit Yadav
|
8ef131745a | Merge branch '4.11' | |
Marc-Aurèle Brothier
|
893a88d225 |
CLOUDSTACK-10105: Use maven standard project structure in all projects (#2283)
Remove maven standard module (which only a few were using) and get ride of maven customization for the projects structure. - moved all directories to src/main/java, src/main/resources, src/main/scripts, src/test/java, src/test/resources - grep scan to search for src/com and src/org left over - grep for <project>/scripts to fix pom.xml configuration - remove custom <build> configuration in pom.xml Signed-off-by: Marc-Aurèle Brothier <m@brothier.org> |