Commit Graph

29297 Commits

Author SHA1 Message Date
Rohit Yadav 8e4644e413 vmware: improve support for disks
- Improve disk chain usage while attaching, migrating disks
- Gets root disk controller based diskDeviceBusName from volume's chain info
- Refactor and move VirtualMachineDiskInfo to cloud-utils
- Allows mixing of scsi controller types
- Fixes a NPE case with map passed as null, for example in case of detach volume
  command
- Use a osdefault translator that allow use of recent os types added (enums of
  which) are not available in the sdk

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-04-18 22:44:18 +05:30
Sverrir Berg 64b72a5c5a Add perl-modules as install dependency for cloudstack-agent
Required to run perl scripts that configure networking for VMs.
That script fails silently if this is not installed.
2016-04-15 13:09:45 +00:00
Sverrir Berg c40a1ae0ed Installing bzip2 since it is required for extracting templates. 2016-04-14 14:16:05 +00:00
Will Stevens 6f703c4cdc Merge pull request #1487 from kollyma/speedup_iptables_prefetch_variables
speedup iptables by prefetching the variables-- This PR is replacing speedup iptables setup #1449
-- Squashing commits and cleanup

PR against 4.7 as discussed with Remi Bergsma. This will speed up the iptables creation on the virtual router.

Testing showed the following:
with current code:
root@kvm704:~# time /usr/share/cloudstack-common/scripts/network/domr/router_proxy.sh vr_cfg.sh 169.254.1.176 -c /var/cache/cloud/VR-12f28879-de7e-44d2-8dbe-b93a04bd3ba4.cfg
real 2m56.401s
user 0m0.012s
sys 0m0.012s

modified version:
root@kvm704:~# time /usr/share/cloudstack-common/scripts/network/domr/router_proxy.sh vr_cfg.sh 169.254.1.176 -c /var/cache/cloud/VR-12f28879-de7e-44d2-8dbe-b93a04bd3ba4.cfg
real 1m35.762s
user 0m0.020s
sys 0m0.004s

* pr/1487:
  speedup iptables by prefetching the variables

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-04-12 10:09:18 -04:00
Will Stevens 8e3d7ee41c Merge pull request #1479 from myENA/9285_exception_log
Cloudstack-9285 exception log additionAfter discussion with @miguelaferreira on the previous PR related to Cloudstack-9285, we decided on adding additional exception logging for this issue.

After adding it, the logs look like this in our lab:

2016-04-07 15:44:03,298 WARN  [cloud.agent.Agent] (Agent-Handler-1:null) (logid:7225632a) NIO Connection Exception com.cloud.utils.exception.NioConnectionException: Connection closed with -1 on reading size.  <<-- new exception logging
2016-04-07 15:44:03,298 INFO  [cloud.agent.Agent] (Agent-Handler-1:null) (logid:7225632a) Attempted to connect to the server, but received an unexpected exception, trying again... << --original logging from previous PR.

* pr/1479:
  Additional exception logging for Cloudstack-9285

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-04-12 08:56:35 -04:00
Wilder Rodrigues 799b9f223d CLOUDSTACK-9287 - Improve test by checking if pvt gw is removed and fix typos 2016-04-12 10:47:04 +02:00
Simon Weller 5f062f1b9e Additional exception logging for Cloudstack-9285 2016-04-11 17:05:45 -05:00
kollyma 0a01e82c23 speedup iptables by prefetching the variables 2016-04-11 20:58:03 +02:00
Will Stevens 2d68893ee6 Merge pull request #1422 from remibergsma/ui-vpc-routers-improvement_47
Improve ordering of fields of VPC router detail tabThe field we use most are now on the top:
- name
- state
- hypervisor
- link local ip
- redundant state

The other fields are nice, but not needed most of the time.

Result:
![screen shot 2016-02-21 at 20 11 49](https://cloud.githubusercontent.com/assets/1630096/13204809/37ce63c0-d8d9-11e5-8861-35c1c1a406cf.png)

Before:
![vpc_router_details_before](https://cloud.githubusercontent.com/assets/1630096/13204811/41510f10-d8d9-11e5-850f-549a3333b063.png)

* pr/1422:
  Improve ordering of fields of VPC router detail tab

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-04-11 08:44:21 -04:00
Will Stevens f33b4a1b68 Merge pull request #1441 from mike-tutkowski/4.7
CLOUDSTACK-9297 - Reworked logic in StorageSystemSnapshotStrategy and XenserverSnapshotStrategyThe ticket this PR fixes was opened because KVM-specific code had been added to the StorageSystemSnapshotStrategy class and that class' canHandle method was only prepared to handle managed storage being used with XenServer (and a case was hit for KVM that triggered a CloudRuntimeException to be thrown).

To solve the problem, I moved the KVM logic to the default snapshot strategy class, which is (unfortunately) named XenserverSnapshotStrategy.

I plan to rename XenserverSnapshotStrategy to something like DefaultSnapshotStrategy in 4.9.

My guess is that when XenserverSnapshotStrategy was originally written, it was written only for XenServer, but has since that time had its usage increased to support other hypervisors (with non-managed storage).

* pr/1441:
  CLOUDSTACK-9297: delete snapshot without id is failing with Unable to determine the storage pool of the snapshot

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-04-11 08:40:17 -04:00
Remi Bergsma f4f9b3ab4e Handle private gateways more reliably 2016-04-10 20:06:44 +02:00
Remi Bergsma 9e1859ee2b Set default networkDomain to empty instead of username
The 10th field of createUserAccount is 'networkDomain' (AccountService.java) and it is set to a var named 'admin', which is the user name.
So, the first user that is created in a domain that links to LDAP, creates the account within the domain, and sets the 'networkDomain' field to the username. All next users are created in the same account.

Then we have the situation that in domain SBP we have a user 'rbergsma' that logs in first, gets an account created and then (unless you override) all VMs started in the SBP domain will have network domain 'rbergsma'. That is highly confusing and not what is should be.

linkDomainToLdap api call has no 'networkDomain' field, so I propose to make this field empty (set it to null). It's a sting and null / empty is allowed.

One can also specify the networkDomain when creating a VPC and also there it is allowed to be null.

When te networkDomain is needed (and is not set in the domain and not in the VPC) it is constructed by using guest.domain.suffix so there always is a netWork domain to be used.

It makes more sense to manually set it on a domain level, or specify it on the VPC and in the final case end up with something that is clearly generated (like cs342cloud.local) rather than the username of someone else.
2016-04-10 19:50:32 +02:00
Wilder Rodrigues 78bbd498e7 CLOUDSTACK-9287 - Fix RVR public interface 2016-04-09 21:14:41 +02:00
Wilder Rodrigues 0e91468964 CLOUDSTACK-9287 - Add integration test to cover the private gateway related changes 2016-04-09 21:14:33 +02:00
Wilder Rodrigues c41edc1fe6 CLOUDSTACK-9287 - Refactor the interface state configuration
- This also refactors the CsAddress in order to offer better readability in a couple of methods.
2016-04-09 21:14:25 +02:00
Wilder Rodrigues 850fb1a557 CLOUDSTACK-9287 - Check if the nic profile has already been removed from a certain router
- In case of redundant VPCs, the ACL items are revoked in the first iteration. Since the econd iteration
     is needed in order to remove the private network, we have to check if the nic profile is gone before trying
     to revoke the ACL items again, which would throw a NPE.
   - Some variable extraction in order to ease debugging.
2016-04-09 21:14:17 +02:00
Remi Bergsma 6a767732f9 CLOUDSTACK-9287 - Bring up the private gw interface on state change to master 2016-04-09 21:14:10 +02:00
Remi Bergsma 057b54aa3e CLOUDSTACK-9287 - Make sure private gw interface is not used for default gw 2016-04-09 21:13:47 +02:00
Wilder Rodrigues 250be376e8 CLOUDSTACK-9287 - Add integration test to cover the private gw interface/mac address issues 2016-04-09 21:13:41 +02:00
Wilder Rodrigues d93b008deb CLOUDSTACK-9287 - Put private gateway interface down on backup router 2016-04-09 21:13:35 +02:00
Wilder Rodrigues f921ec686b CLOUDSTACK-9287 - Generate new mac address if router is redundant and nic profile exists 2016-04-09 21:13:28 +02:00
Remi Bergsma aead9f4ff4 When no zone name is available display a default
Backport from Cosmic
2016-04-07 21:57:17 +02:00
Miguel Ferreira 65cb22216a Add private gateway IP to router initialization config 2016-04-07 21:13:00 +02:00
Remi Bergsma b9feb39e17 apply static routes on change to master state 2016-04-07 20:57:58 +02:00
Remi Bergsma 3636ad1114 lower the time we wait for interfaces to appear
They might never appear.. for example when we have entries in
/etc/cloudstack/ips.json that haven't been plugged yet. Waiting
this long makes everything horribly slow (every vm, interface,
static route, etc, etc, will hit this wait, for every device).
2016-04-07 20:52:33 +02:00
Will Stevens ef115ab1d0 Merge pull request #1332 from syed/swift-download
Add ability to download templates in SwiftThis PR adds the ability to download templates when using Swift as a secondary storage. Uses the "temp_url" feature of Swift so that tempates can be downloaded without authenticaiton.

* pr/1332:
  Add ability to download templates in Swift

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-04-06 14:19:44 -04:00
Syed 52fc2bab69 Add ability to download templates in Swift 2016-04-05 16:40:00 -04:00
Högni Ingimarsson fde0b6aa57 writeIfNotHere requires an array of strings, not a string 2016-03-30 11:17:13 +00:00
Simon Weller 48ce763440 Merge pull request #4 from apache/4.7
4.7 PR
2016-03-24 14:55:34 -05:00
Mike Tutkowski 158d196c05 CLOUDSTACK-9297: delete snapshot without id is failing with Unable to determine the storage pool of the snapshot 2016-03-24 12:42:48 -06:00
Will Stevens 4db1c0125a Merge pull request #1430 from myENA/4.7_cloudstack-9285
CLOUDSTACK-9285 for 4.7.xPer Daan's request, here is a pull request for the 4.7.x release.

* pr/1430:
  Cloudstack 9285 for 4.7.x
  CLOUDSTACK-9285 - Address original on start exception(s) and newline cleanup
  Cloudstack 9285 for 4.7.x

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-03-24 11:01:27 -04:00
Rohit Yadav a11f711062 systemvm: set default umask 022 in injectkeys.sh
The default umask of 0022 is set in Ubuntu and other packages. Set the same
in case of CentOS startup scripts. Use umask 022 in the injectkeys.sh script

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-03-14 12:21:57 +01:00
Simon Weller 454ea4a0b0 Cloudstack-9285 2016-03-08 09:55:24 -06:00
Simon Weller 3683dff493 Cloudstack 9285 for 4.7.x 2016-03-08 09:48:13 -06:00
Simon Weller c69a242897 CLOUDSTACK-9285 - Address original on start exception(s) and newline cleanup 2016-03-08 09:14:07 -06:00
Simon Weller 5b6fbe6aeb Cloudstack 9285 for 4.7.x 2016-03-07 16:23:35 -06:00
Rafael Weingärtner 9610056fa5 Merge pull request #1390 from nitin-maharana/CloudStack-Nitin24_4.7
CLOUDSTACK-9267: String is not localized on create instance wizards.Repro Steps:
==========
1. Setup basic environments as normal.
2. Open a browser, go to Web Console.
3. Upload a iso template.
4. Go to "Instances" page and add new instance.
5. Check the strings on create instance wizards.

Expected Result:
=============
All the strings should be localized on create instance wizards.

Actual Result:
===========
Some strings are not localized on create instance wizards.

Language:
========
JA -> Fail
SC -> Fail

Fix:
===
Fixed the hard coded string.

* pr/1390:
  CLOUDSTACK-9267: String is not localized on create instance wizards.

Signed-off-by: Rafael Weingärtner <rafael@apache.org>
2016-02-25 09:38:25 -03:00
Remi Bergsma d2fdab0750 Improve ordering of fields of VPC router detail tab
The field we use most are now on the top:
- name
- state
- hypervisor
- link local ip
- redundant state

The other fields are nice, but not needed most of the time.
2016-02-21 20:26:46 +01:00
Boris Schrijver 30741a3309 Split the cidr lists so we won't hit the iptables-resture limits 2016-02-05 21:11:59 +01:00
Boris Schrijver 709be45cf0 Check the existence of 'forceencap' parameter before use 2016-02-05 21:06:53 +01:00
Boris Schrijver 18d5cd2855 Do not load previous firewall rules as we replace everyhing anyway 2016-02-05 21:04:06 +01:00
Boris Schrijver eb9706b655 Wait for dnsmasq to finish restart 2016-02-05 12:02:58 +01:00
Boris Schrijver 69e9348900 Remove duplicate spaces, and thus duplicate rules. 2016-02-05 12:02:57 +01:00
Boris Schrijver b857f79fcb Restore iptables at once using iptables-restore instead of calling iptables numerous times 2016-02-05 12:02:57 +01:00
Boris Schrijver 0f75042566 Add iptables copnversion script.
Source: https://raw.githubusercontent.com/sl0/conv/master/iptables_converter.py
2016-02-05 12:02:57 +01:00
Remi Bergsma a243339faa Merge pull request #1386 from remibergsma/fix-del-private-gw-route
CLOUDSTACK-9266: Make deleting static routes in private gw workThe to-be-deleted static routes were removed from the json file, instead of putting them there with revoke=true. The script that parses the json now doesn't find it and thus does not delete it.

Example after adding/removing some:
```
root@r-3-VM:/var/cache/cloud# cat /etc/cloudstack/staticroutes.json
{
    "1.2.3.0/24": {
        "gateway": "172.16.0.1",
        "ip_address": "172.16.0.2",
        "network": "1.2.3.0/24",
        "revoke": true
    },
    "1.2.3.4/32": {
        "gateway": "172.16.0.1",
        "ip_address": "172.16.0.2",
        "network": "1.2.3.4/32",
        "revoke": true
    },
    "1.2.33.3/32": {
        "gateway": "172.16.0.1",
        "ip_address": "172.16.0.2",
        "network": "1.2.33.3/32",
        "revoke": true
    },
    "1.22.2.2/32": {
        "gateway": "172.16.0.1",
        "ip_address": "172.16.0.2",
        "network": "1.22.2.2/32",
        "revoke": true
    },
    "10.1.2.1/32": {
        "gateway": "172.16.0.1",
        "ip_address": "172.16.0.2",
        "network": "10.1.2.1/32",
        "revoke": true
    },
    "10.1.200.0/25": {
        "gateway": "172.16.0.1",
        "ip_address": "172.16.0.2",
        "network": "10.1.200.0/25",
        "revoke": true
    },
    "10.11.12.13/32": {
        "gateway": "172.16.0.1",
        "ip_address": "172.16.0.2",
        "network": "10.11.12.13/32",
        "revoke": true
    },
    "172.16.1.3/32": {
        "gateway": "172.16.0.1",
        "ip_address": "172.16.0.2",
        "network": "172.16.1.3/32",
        "revoke": true
    },
    "172.16.15.14/32": {
        "gateway": "172.16.0.1",
        "ip_address": "172.16.0.2",
        "network": "172.16.15.14/32",
        "revoke": false
    },
    "172.16.17.0/25": {
        "gateway": "172.16.0.1",
        "ip_address": "172.16.0.2",
        "network": "172.16.17.0/25",
        "revoke": false
    },
    "id": "staticroutes"
}
```

This results in:
```
root@r-3-VM:/var/cache/cloud# ip route show
default via 192.168.23.1 dev eth1
169.254.0.0/16 dev eth0  proto kernel  scope link  src 169.254.1.67
172.16.0.0/24 dev eth2  proto kernel  scope link  src 172.16.0.2
172.16.15.14 via 172.16.0.1 dev eth2
172.16.17.0/25 via 172.16.0.1 dev eth2
192.168.23.0/24 dev eth1  proto kernel  scope link  src 192.168.23.4
```

Two static routes left, the rest deleted:
```
172.16.15.14 via 172.16.0.1 dev eth2
172.16.17.0/25 via 172.16.0.1 dev eth2
```

That also matches the UI:

<img width="1327" alt="screen shot 2016-01-30 at 06 34 06" src="https://cloud.githubusercontent.com/assets/1630096/12693933/83e67d80-c71b-11e5-9241-9f478522b7a4.png">

* pr/1386:
  CLOUDSTACK-9266: Make deleting static routes in private gw work

Signed-off-by: Remi Bergsma <github@remi.nl>
2016-02-04 09:13:00 +01:00
Remi Bergsma 556d4362f1 Merge pull request #1383 from remibergsma/slash32routefix
CLOUDSTACK-9264: Make /32 static routes for private gw workStatic routes for private gateways that were /32 failed because the `route` command used had `-net` in it and a `/32` requires `-host` instead. I rewrote it to `ip` commands.

* pr/1383:
  CLOUDSTACK-9264: Make /32 static routes for private gw work

Signed-off-by: Remi Bergsma <github@remi.nl>
2016-02-04 09:12:01 +01:00
Nitin Kumar Maharana 990e42f61b CLOUDSTACK-9272: No option in UI to add GSLB with service type "HTTP"
It supports http also. Added http to the list.
2016-02-03 09:52:13 +05:30
Nitin Kumar Maharana 7c7bee8bba CLOUDSTACK-9270: UI alignment gone bad in multiple places - VM Instance, Network, Egress rules
Now it breaks into two lines once the word goes out of the box.
Fixed the advanced search field issue.
2016-02-03 01:09:36 +05:30
Nitin Kumar Maharana f2d3cc893c CLOUDSTACK-9267: String is not localized on create instance wizards.
Fixed the hard coded string.
Added _l() to dictioanry keys.
2016-02-03 00:28:28 +05:30