Commit Graph

28194 Commits

Author SHA1 Message Date
Remi Bergsma 64ff67da55 Merge pull request #654 from DaanHoogland/CLOUDSTACK-8656
Cloudstack 8656: do away with more silently ignoring exceptions.a lot of messages added.
some restructuring for test exception assertions and try-with-resource blocks

* pr/654: (29 commits)
  CLOUDSTACK-8656: more logging instead of sysout
  CLOUDSTACK-8656: use catch block for validation
  CLOUDSTACK-8656: class in json specified not found
  CLOUDSTACK-8656: removed unused classes
  CLOUDSTACK-8656: restructure of tests
  CLOUDSTACK-8656: reorganise sychronized block
  CLOUDSTACK-8656: restructure tests to ensure exception throwing
  CLOUDSTACK-8656: validate the throwing of ServerApiException
  CLOUDSTACK-8656: logging ignored exceptions
  CLOUDSTACK-8656: try-w-r removes need for empty catch block
  CLOUDSTACK-8656: try-w-r instead of clunckey close-except
  CLOUDSTACK-8656: deal with empty SQLException catch block by try-w-r
  CLOUDSTACK-8656: unnecessary close construct removed
  CLOUDSTACK-8656: message about timed buffer logging
  CLOUDSTACK-8656: message about invalid number from store
  CLOUDSTACK-8656: move cli test tool to separate file
  CLOUDSTACK-8656: exception is the rule for some tests
  CLOUDSTACK-8656: network related exception logging
  CLOUDSTACK-8656: reporting ignored exceptions in server
  CLOUDSTACK-8656: log in case we are on a platform not supporting UTF8
  ...

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-08-14 21:38:49 +02:00
Daan Hoogland b6f1d29e64 CLOUDSTACK-8656: more logging instead of sysout 2015-08-14 17:28:29 +02:00
Daan Hoogland adcd0c0c0d CLOUDSTACK-8656: use catch block for validation 2015-08-14 17:25:04 +02:00
Daan Hoogland 384a2e9cee CLOUDSTACK-8656: class in json specified not found 2015-08-14 16:29:21 +02:00
Daan Hoogland 237f922ff7 CLOUDSTACK-8656: removed unused classes 2015-08-14 16:14:17 +02:00
Daan Hoogland bb8facc37c CLOUDSTACK-8656: restructure of tests 2015-08-14 16:07:12 +02:00
Daan Hoogland 882de20e1c CLOUDSTACK-8656: reorganise sychronized block 2015-08-14 16:03:17 +02:00
Daan Hoogland 07bee3149c CLOUDSTACK-8656: restructure tests to ensure exception throwing 2015-08-14 15:43:54 +02:00
Daan Hoogland 2eee6aa999 CLOUDSTACK-8656: validate the throwing of ServerApiException 2015-08-14 15:13:39 +02:00
Daan Hoogland de2d78b18b CLOUDSTACK-8656: logging ignored exceptions 2015-08-14 15:11:38 +02:00
Daan Hoogland 6f2f81c4ac CLOUDSTACK-8656: try-w-r removes need for empty catch block 2015-08-14 14:43:03 +02:00
Daan Hoogland a923abf178 CLOUDSTACK-8656: try-w-r instead of clunckey close-except 2015-08-14 14:38:58 +02:00
Daan Hoogland 6c7ab8a5a5 CLOUDSTACK-8656: deal with empty SQLException catch block by try-w-r 2015-08-14 14:09:38 +02:00
Daan Hoogland 6227c44ca4 CLOUDSTACK-8656: unnecessary close construct removed 2015-08-14 13:59:59 +02:00
Daan Hoogland 08e6e1633c CLOUDSTACK-8656: message about timed buffer logging 2015-08-14 13:29:35 +02:00
Daan Hoogland 4213a22127 CLOUDSTACK-8656: message about invalid number from store 2015-08-14 13:21:46 +02:00
Daan Hoogland 3229d96015 CLOUDSTACK-8656: move cli test tool to separate file 2015-08-14 13:15:11 +02:00
Daan Hoogland f203acc074 CLOUDSTACK-8656: exception is the rule for some tests 2015-08-14 13:07:48 +02:00
Remi Bergsma 05a29f01b4 Merge pull request #693 from remibergsma/s2svpn-fixes
Fix site-to-site VPN featureThis is work done together with @jayapalu on fixing the site2site VPN. The first part was done in PR #690 by @jayapalu. On top of that, some other fixes were needed and those are added in this PR. It made sense to make a new PR which includes all fixes so we can actually test it.

The original PR #690 is already merged into this one, so can be closed. Since the commit ids are kept the same, merging this will close both.

I closely compared the 4.4/4.5 implementation with the new 4.6 one. I did not only make it work, but also added some security improvements (some of which were also in 4.4/4.5). I noticed the pre shared key was being logged, so removed that as well.

This is how I tested and verified it:
https://github.com/schubergphilis/MCT-shared/tree/master/helper_scripts/cloudstack/vpn_tests
When I have some time available, I'll write a Marvin test for it that we can include in the repo.

It now works(tm) with one manual step due to CLOUDSTACK-8685:
We need a default gateway before site-to-site VPN will actually work. It will connect, but not forward packets. The reason for this, is due to the iptables setup. VM1 has router1 as gateway, but router1 does not know the route to VM2 so it will give up. With a default gateway, the packets are about to be forwarded to the default gateway but when they reach eth1 the public nic, iptables kicks in, does some magic and forwards it through the ipsec tunnel. So, you need a default gw set to upstream.

Workaround for now is setting the route manually:
``route add default gw 1.2.3.4``  or  ``ip route add default via 1.2.3.4``

In other words, we need to fix CLOUDSTACK-8685 soon, too.

Thanks to @snuf @jayapalu!

@jayapalu @snuf could you please review this?

* pr/693:
  do not log sensitive site-to-site VPN PSK
  tighten security of site-to-site VPN
  CLOUDSTACK-8730: fix s2s iptables rules and ipsec config
  CLOUDSTACK-8710: Fixed applying iptables rules for s2s vpn

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-08-14 13:05:53 +02:00
Remi Bergsma 0fcc7297e0 Merge pull request #680 from borisroman/CLOUDSTACK-8580
Interface changes related to CLOUDSTACK-8580See issue CLOUDSTACK-8580 and individual commits.

* pr/680:
  Made interface changes related to CLOUDSTACK-8580

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-08-14 11:47:36 +02:00
Boris Schrijver 655fcc924f Made interface changes related to CLOUDSTACK-8580
Added responses to ListCapabilities to reflect CLOUDSTACK-8580 changes.

This to add these options to the gui. See issue CLOUDSTACK-8580.

3be14e978a

Removed comments as proposed by Daan Hoogland.
2015-08-14 11:13:21 +02:00
Milamber 72e55029ef Improve debug message when the scheduler prepare to snapshot (volume 'id' and add volume name)
Signed-off-by: Milamber <milamber@apache.org>
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

This closes #676

(cherry picked from commit 606afe341d)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-08-14 14:12:46 +05:30
Rohit Yadav f7778194f0 tools: fix build_asf.sh release script to remove -SNAPSHOT from bunch of files
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit cc63611cc2)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-08-14 13:55:18 +05:30
Rohit Yadav 952a853392 CLOUDSTACK-8696: Port Rajani's regions fix to 451to452 sql upgrade path
Upgrade paths from 4.5.1 exists to both 4.5.2, and 4.6.0. Since 4.5.2 and 4.6.0
are not release, and the bug affects 4.5 branch; this patch aims to port that
fix from master to both 4.5/master branches.

Ported from commit b6a7804

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-08-14 13:16:58 +05:30
Remi Bergsma 7ddec661ca Merge pull request #690 from jayapalu/vpn
CLOUDSTACK-8710: Fixed applying iptables rules for s2s vpn
@remibergsma @wilderrodrigues
Moved applying iptables rules apply after vpn configuration so that vpn specific rules also get applied

* pr/690:
  CLOUDSTACK-8710: Fixed applying iptables rules for s2s vpn

This closes #690

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-08-14 09:11:21 +02:00
Remi Bergsma 4f8ab51f7f do not log sensitive site-to-site VPN PSK
Logging before:
2015-08-12 16:30:07,126 Searching for 192.168.23.6  and replacing with 192.168.23.6 192.168.23.5: PSK "preSharedKey"

Logging after:
2015-08-12 16:30:07,126 Searching for 192.168.23.6  and replacing with 192.168.23.6 192.168.23.5: PSK "****"
2015-08-14 09:07:25 +02:00
Remi Bergsma 9b97719c5c tighten security of site-to-site VPN
It was like this in 4.4 and 4.5
2015-08-14 09:05:59 +02:00
Remi Bergsma 382458317e CLOUDSTACK-8730: fix s2s iptables rules and ipsec config
For site2site VPN to work, we need a default gateway to be set.
See CLOUDSTACK-8685
2015-08-14 09:02:21 +02:00
Rohit Yadav 869a83f8f8 plugins: fix test case, interface usage and checkstyle import issue
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-08-14 12:23:50 +05:30
Rohit Yadav a3e6942e85 CLOUDSTACK-8701: Add unit test for SAML2AuthManagerImpl
(cherry picked from commit 5f06ef77d8)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

This closes #650
2015-08-14 12:15:47 +05:30
Rohit Yadav 25ccf4126d CLOUDSTACK-8701: Add listandswitchsamlaccount API test and add boundary checks
- Adds unit test for ListAndSwitchSAMLAccountCmd
- Checks and logs in user only if they are enabled
- If saml user switches to a locked account, send appropriate error message

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit b30977911d)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-08-14 12:00:04 +05:30
Rohit Yadav fcbee609b4 engine: fix accountState setter name
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 3c9d1fdf5d)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-08-14 11:59:47 +05:30
Rohit Yadav 1d190fd0bb utils: add HttpUtils test
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 0fcd9cad1e)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-08-14 11:59:31 +05:30
Rohit Yadav da72ba6e1b CLOUDSTACK-8701: Add administrative contact block as per SAML IDP expectations
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 1e3e674514)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-08-14 11:59:16 +05:30
Rohit Yadav 1ec4d0155a CLOUDSTACK-8701: Allow SAML users to switch accounts
SAML authorized accounts might be across various domains, this allows for
switching of accounts only in case of SAML authenticated user accounts across
other accounts with the same SAML uid/username.

Moves the previous switch account logic to its own ui-custom module

(cherry picked from commit 1065661cd5)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

Conflicts:
	plugins/user-authenticators/saml2/src/org/apache/cloudstack/api/command/SAML2LoginAPIAuthenticatorCmd.java
	ui/index.jsp
2015-08-14 11:58:44 +05:30
Rohit Yadav 89f47ece3d CLOUDSTACK-8702: Add/refactor sessionkey checking code to HttpUtils
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit cb7dd7b27d)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-08-14 11:56:55 +05:30
Rohit Yadav 5d29b63cfa CLOUDSTACK-8613, CLOUDSTACK-6301: Dump KVM domain XML with secure flag
When dumping XML use appropriate flags:

1, VIR_DOMAIN_XML_SECURE (dump security sensitive information too)
8, VIR_DOMAIN_XML_MIGRATABLE (dump XML suitable for migration)

Source:
https://libvirt.org/html/libvirt-libvirt-domain.html#virDomainXMLFlags

This fixes CVE 2015-3252: VNC password lost during VM migration across KVM
hosts. The issue is also seen when a VM is rebooted.

(cherry picked from commit cb2aca7516)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-08-14 10:57:36 +05:30
John Burwell ef44c7d305 CLOUDSTACK-8566: Strips the username and password credentials from host details
... map returned by the following API calls by filtering these fields from the
details attribute in the HostResponse class:

  * listHosts
  * addHost
  * cancelHostMaintenance
  * listHosts
  * prepareHostForMaintenance
  * reconnectHost
  * updateHost

This fix addresses CVE 2015-3251.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 3a48171bd8)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-08-14 10:49:40 +05:30
Daan Hoogland ff66175f55 Merge pull request #681 from DaanHoogland/coverity-regressions
Coverity regressions per 10 aug 2015Not all are in here, the db upgrade code seems to stay the main pitfall.

* pr/681:
  coverity 1315775: proper getting of networkLabel
  coverity 1315774: improvement of code to negate false positive

Signed-off-by: Daan Hoogland <daan@onecht.net>
2015-08-13 16:25:01 +02:00
Boris Schrijver aa7f8e57c5 Added unit tests for CLOUDSTACK-8133
Tests will confirm the behaviour of the newly added response fields of listSecurityGroups.

Signed-off-by: Wido den Hollander <wido@widodh.nl>

This closes #679
2015-08-13 13:47:38 +02:00
Boris Schrijver 03f48872d6 CLOUDSTACK-8133 Added Virtualmachine count and ID's to listSecurityGroups response.
See issue CLOUDSTACK-8133 for more information.

Added null check by comment of Koushik Das.

Added brackets by comment of Wido den Hollander.

Removed a call to findById() by comment of Koushik Das.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2015-08-13 13:47:38 +02:00
shweta agarwal 792c27c9bd test case automated for list template pagination
additional verification added
This closes #646
2015-08-13 15:51:16 +05:30
Remi Bergsma af7e9b8dec Merge pull request #668 from pritisarap12/CLOUDSTACK-8693-Adding-missing-code-in-testpath_same_vm_name.py_testpatha
CLOUDSTACK-8693-Adding-missing-code-in-testpath_same_vm_name.py_testpath-Adding "cls.hypervisor = cls.testClient.getHypervisorInfo()"
-Fixed pep8 issues

* pr/668:
  CLOUDSTACK-8693-Adding-missing-code-in-testpath_same_vm_name.py_testpatha

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-08-13 12:00:35 +02:00
Jayapal da9e757b8e CLOUDSTACK-8710: Fixed applying iptables rules for s2s vpn 2015-08-13 14:07:12 +05:30
Priti Sarap 6fd491b1ce CLOUDSTACK-8693-Adding-missing-code-in-testpath_same_vm_name.py_testpatha 2015-08-13 13:11:50 +05:30
Mike Tutkowski 8fcc76e7ff Fixing a FindBugs issue related to PR 674 2015-08-12 22:57:59 -06:00
Mike Tutkowski 55ae168cdd Merge pull request #674 from Mike Tutkowski (mike.tutkowski@solidfire.com)
* pr/674:
  getUsedBytes should query the SolidFire cluster to acquire the size of the given volume if there is no volume_details info for that volume (and then create a volume_details row for this volume so we don't have to make that cluster call for this purpose again)

Signed-off-by: Mike Tutkowski
Signed-off-by: Mike Tutkowski <mike.tutkowski@solidfire.com>
2015-08-12 12:48:36 -06:00
Mike Tutkowski bcfbc6d73c getUsedBytes should query the SolidFire cluster to acquire the size of the given volume if there is no volume_details info for that volume (and then create a volume_details row for this volume so we don't have to make that cluster call for this purpose again) 2015-08-12 12:24:17 -06:00
Daan Hoogland 1ab3b96594 coverity 1315775: proper getting of networkLabel 2015-08-12 01:10:31 +02:00
Daan Hoogland a3ae8e0645 coverity 1315774: improvement of code to negate false positive 2015-08-12 01:04:06 +02:00