Adds three new optional CLI flags to nasbackup.sh:
-M|--mode <full|incremental>
--bitmap-new <name> (checkpoint to create with this backup)
--bitmap-parent <name> (incremental: parent bitmap to read changes since)
--parent-path <path> (incremental: parent backup file for rebase)
Behavior:
- When -M is omitted, behavior is unchanged (legacy full-only, no checkpoint
created), so existing callers are not affected.
- With -M full + --bitmap-new, a full backup is taken AND a libvirt
checkpoint of that name is registered atomically (via backup-begin's
--checkpointxml), giving the next incremental its starting bitmap.
- With -M incremental, libvirt's <incremental> element references the
parent bitmap; only changed blocks are written. After completion,
qemu-img rebase wires the new file to its parent so the chain on the
NAS is self-describing for restore.
- Stopped VMs cannot use backup-begin; if -M incremental is requested
while VM is stopped, the script falls back to a full and emits
INCREMENTAL_FALLBACK= on stderr so the orchestrator can record it
correctly in the chain.
- The script echoes BITMAP_CREATED=<name> on success so the Java caller
can store it under backup_details (NASBackupChainKeys.BITMAP_NAME).
Works across local file, NFS-file, and LINSTOR primary storage. Ceph RBD
running-VM support is a pre-existing limitation of this script, not
affected by this change.
Refs: apache/cloudstack#12899
NASBackupChainKeys defines the keys this provider stores under the
existing backup_details kv table (parent_backup_id, bitmap_name,
chain_id, chain_position, type). This keeps the backups table
provider-agnostic per the RFC review.
nas.backup.full.every is a zone-scoped ConfigKey that controls how
often a full backup is taken; the remaining backups in the cycle are
incremental. Counts backups (not days), so it works for hourly,
daily, and ad-hoc schedules. Default 10. Set to 1 to disable
incrementals (every backup is full).
Refs: apache/cloudstack#12899
Adds the design document for incremental NAS backups using QEMU dirty
bitmaps and libvirt's backup-begin API. Reduces daily backup storage
80-95% for large VMs.
Refs: apache/cloudstack#12899
* Move logs for values of the migration settings out of the loop
* Apply suggestions from code review
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
---------
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
Fixes an issue in NsxResource.executeRequest where Network.Service
comparison failed when DeleteNsxNatRuleCommand was executed in a
different process. Due to serialization/deserialization, the
deserialized Network.Service instance was not equal to the static
instances Network.Service.StaticNat and Network.Service.PortForwarding,
causing the comparison to always return false.
Co-authored-by: Andrey Volchkov <avolchkov@playtika.com>
(cherry picked from commit 30dd234b00)
* Fix static routes to be added to PBR tables in VPC routers
Static routes were only being added to the main routing table, but
policy-based routing (PBR) is active on VPC routers. This caused
traffic coming in from specific interfaces to not find the static
routes, as they use interface-specific routing tables (Table_ethX).
This fix:
- Adds a helper method to find which interface a gateway belongs to
by matching the gateway IP against configured interface subnets
- Modifies route add/delete operations to update both the main table
and the appropriate interface-specific PBR table
- Uses existing CsAddress databag metadata to avoid OS queries
- Handles both add and revoke operations for proper cleanup
- Adds comprehensive logging for troubleshooting
Fixes#12857
* Add iptables FORWARD rules for nexthop-based static routes
When static routes use nexthop (gateway) instead of referencing a
private gateway's public IP, the iptables FORWARD rules were not
being generated. This caused traffic to be dropped by ACLs.
This fix:
- Adds a shared helper CsHelper.find_device_for_gateway() to determine
which interface a gateway belongs to by checking subnet membership
- Updates CsStaticRoutes to use the shared helper instead of duplicating
the device-finding logic
- Modifies CsAddress firewall rule generation to handle both old-style
(ip_address-based) and new-style (nexthop-based) static routes
- Generates the required FORWARD and PREROUTING rules for nexthop routes:
* -A PREROUTING -s <network> ! -d <interface_ip>/32 -i <dev> -j ACL_OUTBOUND_<dev>
* -A FORWARD -d <network> -o <dev> -j ACL_INBOUND_<dev>
* -A FORWARD -d <network> -o <dev> -m state --state RELATED,ESTABLISHED -j ACCEPT
Fixes the second part of #12857
* network matching grep fix, don't let 1.2.3.4/32 match 11.2.3.4/32
* initial attempt at network.loadbalancer.haproxy.idle.timeout implementation
* implement test cases
* move idleTimeout configuration test to its own test case
`cursor` field when more pages are available. The previous implementation only
fetched the first page and ignored pagination.
This change updates the list retrieval flow to:
- follow the `cursor` chain until no further pages exist
- accumulate items from all pages
- return a single merged result to the caller
This ensures that list operations return the complete dataset rather than just
the first page.
Co-authored-by: Andrey Volchkov <avolchkov@playtika.com>
* Fix VPC restart with multi-CIDR networks: handle comma-separated CIDR in NetworkVO.equals()
When a network has multiple CIDRs (e.g. '192.168.2.0/24,160.0.0.0/24'),
NetworkVO.equals() passes the raw comma-separated string to
NetUtils.isNetworkAWithinNetworkB() which expects a single CIDR,
causing 'cidr is not formatted correctly' error during VPC restart
with cleanup=true.
Extract only the first CIDR value before passing to NetUtils.
* Fix root cause: skip CIDR/gateway updates for Public traffic type networks
addCidrAndGatewayForIpv4/Ipv6 (introduced by PR #11249) was called for all
network types without checking if the network is Public. This caused
comma-separated CIDRs to be stored on Public networks, which then triggered
'cidr is not formatted correctly' errors during VPC restart.
Add TrafficType.Public guard in both the VLAN creation (addCidr) and
VLAN deletion (removeCidr) paths in ConfigurationManagerImpl.
* Sanitize legacy network-level addressing fields for Public networks
---------
Co-authored-by: dahn <daan@onecht.net>
Changes to check resource limits with reservations for the following
resource types:
- backup
- backup_storage
- bnucket
- object_storage
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
* kvm: fix wrong CheckVirtualMachineAnswer when vm does not exist
* kvm: add LibvirtCheckVirtualMachineCommandWrapperTest
Co-authored-by: dahn <daan.hoogland@gmail.com>
James Peru
Henrique Sato
Suresh Kumar Anaparti
dahn
Nicolas Vazquez
Wei Zhou
Daniil Zhyliaiev
Brad House - Nexthop
Brad House
Abhishek Kumar
Daan Hoogland
Fabricio Duarte
Erik Böck
Harikrishna
julien-vaz
Jtolelo
sandeeplocharla
Abhisar Sinha
João Jandre
Manoj Kumar
Abhisar Sinha
poddm
Bernardo De Marco Gonçalves
Vishesh