Cloudstack Source code
Go to file
Wei Zhou 67b60385af
Apple FR68: New database encryption cipher (#214)
* Rough start swapping DB Encryption, add CLI PoC

* Enhance EncryptionCLI to have command line parsing

* Refactor new encryption behind AeadBase64Encryptor for every use

* Add comment about encryption passwords

* EncryptionSecretKeyChanger - use reflection to find all encrypted tables

Over the years this hasn't been updated properly. Use reflection to find
the tables with encrypted fields. This will also ensure any plugins in
the classpath that add tables will get their encrypted fields updated as well.

Table vpn_users has encrypted columns [password]
Table sslcerts has encrypted columns [password, key]
Table user_view has encrypted columns [secret_key]
Table account_details has encrypted columns [value]
Table domain_details has encrypted columns [value]
Table s2s_customer_gateway has encrypted columns [ipsec_psk]
Table ucs_manager has encrypted columns [password]
Table vm_instance has encrypted columns [vnc_password]
Table passphrase has encrypted columns [passphrase]
Table keystore has encrypted columns [key]
Table external_stratosphere_ssp_credentials has encrypted columns [password]
Table storage_pool has encrypted columns [user_info]
Table remote_access_vpn has encrypted columns [ipsec_psk]
Table user has encrypted columns [secret_key]
Table oobm has encrypted columns [password]

* Apple FR68: add new class CloudStackEncryptor

* Apple FR68: add interface com.cloud.utils.crypt.Encryptor

* Apple FR68: update com.cloud.utils.EncryptionUtil

* Apple FR68: add cloudstack-utils.jar to cloudstack-common package

* Apple FR68: use cloudstack-utils.jar in scripts

* Apple FR68: revert replace.properties to original version

* Apple FR68: update EncryptionSecretKeyChanger

* Apple FR68: Add EncryptorVersion to CloudStackEncryptor

* Apple FR68: Update com.cloud.utils.crypt.EncryptionCLI

* Apple FR68: Remove check on EncryptionSecretKeyChecker.useEncryption in CloudStackEncryptor

* Apple FR68: update EncryptionSecretKeyChanger part2

* Apple FR68: update EncryptionSecretKeyChanger part3 (force update)

* Apple FR68: move cloud-migrate-databases.in to deprecated and recreate it with java command

* Apple FR68: update EncryptionSecretKeyChanger part4 (add skip-database-migration)

* Apple FR68: set encryptor in first encryption in CloudStackEncryptor

* Apple FR68: save db.cloud.encryptor.version in db.properties

* Apple FR68: update EncryptionSecretKeyChanger part4 (clear db.cloud.encryptor.version)

* Apple FR68: load and save db.cloud.encryptor.version in db.properties

* Apple FR68: Add caller class name in debug messages

* Apple FR68: consider non-exist tables and columns

* Apple FR68: skip tables if no data exists

* Apple FR68: remove GeneralSecurityException from code

* Apple FR68: hide value with Asterisks in CloudStackEncryptor

* Apple FR68: log an error message when fail to load 'init'

* Apple FR68: remove setup/bindir/cloud-migrate-databases.deprecated.in which I think it is not needed

* Apple FR68: add new encryptor version to EncryptionSecretKeyChanger

* Apple FR68: use System.exit(1) in EncryptionSecretKeyChanger

* Apple FR68: check arguments in cloudstack-migrate-databases

* Apple FR68: remove all org.jasypt.* in code

* Apple FR68: initilize database encryptors by getting 'init'

* Apple FR68: migrate server.properties

* Apple FR68: load new management key from environment variable CLOUD_SECRET_KEY_NEW

* Apple FR68: fix unable to load 'init' in fresh installation

* Apple FR68: fix 'Rolling back the transaction' in txn.close

* Apple FR68: improve logging in cloudstack-migrate-databases

* Apple FR68: hide value with Asterisks in other encryptors

* Apple FR68: System.exit(1) if fail to migrate server.properties

* Apple FR68: migrate values from cluster_details,user_vm_details,etc

* Apple FR68: refactor EncryptionSecretKeyChanger

* Apple FR68: update user_vm_deploy_as_is_details values

* Apple FR68: update image_store.url (if protocol is cifs) and storage_pool.path (if pool_type is SMB)

* Apple FR68: minor improvement EncryptionSecretKeyChanger

* Apple FR68: add unit test EncryptionSecretKeyChangerTest

* Apple FR68: support encryption type 'env' in cloudstack-setup-databases to get env "CLOUD_SECRET_KEY" before passed value

* Apple FR68: rename Encryptor to Base64Encryptor

* Apple FR68: Backport community PR 6542

* Apple FR68: code optimization

* Apple FR68: use Options and StringUtils

* Apple FR68: add license headers

* Apple FR68: refactor CloudStackEncryptor as per Daan's review

* Apple FR68: refactor DatabaseUpgradeChecker as per Daan's review

* Apple FR68: show error message in usage.log if fail to get encrypted configurations

* Apple FR68: load new MS key from env before migration

* Apple FR68: return 1 if fail to parse arguments of EncryptionCLI

* Apple FR68: fix code smells

* Apple FR68: fix code smells (part2)

* Apple FR68: revert FOOTER of cloudstack-migrate-databases to use \n

* Apple FR68: update help message of cloudstack-setup-databases

* Apple FR68: fix code smells (part3)

* Apple FR68: make changes as per suggestions

* Apple FR68: migrate database if new encryptor version is set to different

Testing result: (assume db.cloud.encryptor.version=V1)

(1) migrate only db.properties (same db key, same db encryptor version)
Command: /usr/bin/cloudstack-migrate-databases -m mgmtkey -d dbkey -n newmgmtkey -v V1
Changes: db.cloud.encrypt.secret is encrypted by V2 (always)
         db.cloud.encryptor.version=V1
         cloudstack database is not migrated

(2) migrate only db.properties (same db key, new db encryptorversion)
Command: /usr/bin/cloudstack-migrate-databases -m mgmtkey -d dbkey -n newmgmtkey -v V2 --skip-database-migration
Changes: db.cloud.encrypt.secret is encrypted by V2 (always)
         db.cloud.encryptor.version=V2
         cloudstack database is not migrated (mostly on secondary management servers)

(3) migrate only db.properties (same db key, db encryptor version is not set)
Command: /usr/bin/cloudstack-migrate-databases -m mgmtkey -d dbkey -n newmgmtkey
Changes: db.cloud.encrypt.secret is encrypted by V2 (always)
         db.cloud.encryptor.version=V1
         cloudstack database is not migrated

(4) migrate only db.properties (different db key, same db encryptor version)
Command: /usr/bin/cloudstack-migrate-databases -m mgmtkey -d dbkey -n newmgmtkey -e newdbkey -v V1 --skip-database-migration
Changes: db.cloud.encrypt.secret is encrypted by V2 (always)
         db.cloud.encryptor.version=V1
         cloudstack database is not migrated (mostly on secondary management servers)

(5) migrate only db.properties (different db key, new db version)
Command: /usr/bin/cloudstack-migrate-databases -m mgmtkey -d dbkey -n newmgmtkey -e newdbkey -v V2 --skip-database-migration
Changes: db.cloud.encrypt.secret is encrypted by V2 (always)
         db.cloud.encryptor.version=V2
         cloudstack database is not migrated (mostly on secondary management servers)

(6) migrate only db.properties (different db key, db encryptor version is not set)
Command: /usr/bin/cloudstack-migrate-databases -m mgmtkey -d dbkey -n newmgmtkey -e newdbkey --skip-database-migration
Changes: db.cloud.encrypt.secret is encrypted by V2 (always)
         db.cloud.encryptor.version=V1
         cloudstack database is not migrated (mostly on secondary management servers)

(7) migrate db.properties and database (same db key, same db encryptor version)
Command: /usr/bin/cloudstack-migrate-databases -m mgmtkey -d dbkey -n newmgmtkey -v V1 --force-database-migration
Changes: db.cloud.encrypt.secret is encrypted by V2 (always)
         db.cloud.encryptor.version=V1
         cloudstack database is migrated using encryptor V1

(8) migrate db.properties and database (same db key, new db encryptor version)
Command: /usr/bin/cloudstack-migrate-databases -m mgmtkey -d dbkey -n newmgmtkey -v V2
Changes: db.cloud.encrypt.secret is encrypted by V2 (always)
         db.cloud.encryptor.version=V2
         cloudstack database is migrated using encryptor V2

(9) migrate db.properties and database (same db key, db encryptor version is not set)
Command: /usr/bin/cloudstack-migrate-databases -m mgmtkey -d dbkey -n newmgmtkey --force-database-migration
Changes: db.cloud.encrypt.secret is encrypted by V2 (always)
         db.cloud.encryptor.version=V1
         cloudstack database is migrated using encryptor V1

(10) migrate db.properties and database (different db key, same db encryptor version)
Command: /usr/bin/cloudstack-migrate-databases -m mgmtkey -d dbkey -n newmgmtkey -e newdbkey -v V1
Changes: db.cloud.encrypt.secret is encrypted by V2 (always)
         db.cloud.encryptor.version=V1
         cloudstack database is migrated using encryptor V1

(11) migrate db.properties and database (different db key, new db encryptor version)
Command: /usr/bin/cloudstack-migrate-databases -m mgmtkey -d dbkey -n newmgmtkey -e newdbkey -v V2
Changes: db.cloud.encrypt.secret is encrypted by V2 (always)
         db.cloud.encryptor.version=V2
         cloudstack database is migrated using encryptor V2

(12) migrate db.properties and database (different db key, db encryptor version is not set)
Command: /usr/bin/cloudstack-migrate-databases -m mgmtkey -d dbkey -n newmgmtkey -e newdbkey
Changes: db.cloud.encrypt.secret is encrypted by V2 (always)
         db.cloud.encryptor.version=V1
         cloudstack database is migrated using encryptor V1

* smoke test: fix test_primary_storage.py

* smoke test: Do NOT run tests in test_primary_storage.py in parallel

This also fixes an issue in detachvolume

'Failed to detach volume Test Volume-yyyyyy from VM VM-zzzzzz; com.cloud.exception.InternalErrorException: Could not detach volume. Probably the VM is in boot state at the moment'

* Update PR7003: rename method

---------

Co-authored-by: Marcus Sorensen <mls@apple.com>
2023-01-31 13:18:28 +01:00
agent FR65: Secure KVM VNC connection using CA framework (#217) 2023-01-19 08:35:16 -03:00
api Backport community fix for #210 (#222) 2023-01-30 08:19:48 -03:00
build CLOUDSTACK-10012: Migrate to Embedded Jetty 2017-11-03 23:57:25 +05:30
client Apple FR68: New database encryption cipher (#214) 2023-01-31 13:18:28 +01:00
cloud-cli packaging: Adding Centos8, Ubuntu 20.04, XCPNG8.1 Support (#4068) 2020-08-17 16:28:30 +05:30
core Fix UEFI detection on KVM and prevent deployments on non UEFI enabled hosts (#6423) (#221) 2023-01-27 15:53:53 +05:30
debian Apple FR68: New database encryption cipher (#214) 2023-01-31 13:18:28 +01:00
deps deps: fix pbm url download (#4440) 2020-11-03 18:13:31 +05:30
developer Updating pom.xml version numbers for release 4.16.1.0 2022-02-25 19:01:16 +05:30
engine Apple FR68: New database encryption cipher (#214) 2023-01-31 13:18:28 +01:00
framework Apple FR68: New database encryption cipher (#214) 2023-01-31 13:18:28 +01:00
packaging Apple FR68: New database encryption cipher (#214) 2023-01-31 13:18:28 +01:00
plugins Fix UEFI detection on KVM and prevent deployments on non UEFI enabled hosts (#6423) (#221) 2023-01-27 15:53:53 +05:30
python FR65: Secure KVM VNC connection using CA framework (#217) 2023-01-19 08:35:16 -03:00
quickcloud Updating pom.xml version numbers for release 4.16.1.0 2022-02-25 19:01:16 +05:30
scripts Apple FR68: New database encryption cipher (#214) 2023-01-31 13:18:28 +01:00
server Apple FR68: New database encryption cipher (#214) 2023-01-31 13:18:28 +01:00
services Backport community fix for #210 (#222) 2023-01-30 08:19:48 -03:00
setup Apple FR68: New database encryption cipher (#214) 2023-01-31 13:18:28 +01:00
systemvm FR65: Secure KVM VNC connection using CA framework (#217) 2023-01-19 08:35:16 -03:00
test Apple FR68: New database encryption cipher (#214) 2023-01-31 13:18:28 +01:00
tools Apple FR68: New database encryption cipher (#214) 2023-01-31 13:18:28 +01:00
ui Apple FR66 - Host Control Plane Status (#213) 2023-01-04 13:01:10 +01:00
usage Apple FR68: New database encryption cipher (#214) 2023-01-31 13:18:28 +01:00
utils Apple FR68: New database encryption cipher (#214) 2023-01-31 13:18:28 +01:00
vmware-base Updating pom.xml version numbers for release 4.16.1.0 2022-02-25 19:01:16 +05:30
.dockerignore CLOUDSTACK-3049: Implemented role update for account. (#3058) 2019-01-25 10:02:56 -02:00
.gitignore CKS Enhancements and SystemVM template upgrade improvements (#5863) 2022-02-15 18:27:14 +05:30
.java-version cloudstack: add JDK11 support (#3601) 2020-02-12 12:58:25 +05:30
.python-version python3: Migrate Marvin and smoketests to python3 (#4727) 2021-05-04 23:19:37 +05:30
.travis.yml OVS/GRE: bug fixes (#5446) 2021-10-03 14:47:52 +05:30
CHANGES.md dummy commit to for PR close #95 2015-03-08 11:47:29 -04:00
CONTRIBUTING.md cloudstack: make code more inclusive 2021-06-08 15:47:20 +05:30
INSTALL.md INSTALL: updated maven download link (#4709) 2021-02-19 14:13:57 +05:30
ISSUE_TEMPLATE.md cloudstack: make code more inclusive 2021-06-08 15:47:20 +05:30
LICENSE CLOUDSTACK-10105: Use maven standard project structure in all projects (#2283) 2018-01-20 03:19:27 +05:30
LICENSE.header Add license check plugin to root pom. Disabled for the entire project, except for poms that override it (opendaylight, nvp) 2014-01-22 11:22:42 +01:00
NOTICE Update NOTICE to account for updated jQuery version 2014-08-26 13:13:29 -07:00
PULL_REQUEST_TEMPLATE.md cloudstack: make code more inclusive 2021-06-08 15:47:20 +05:30
PendingReleaseNotes Add a PendingRelease notes file (#2723) 2018-07-03 21:17:41 +02:00
README.md Update README.md (#5492) 2021-09-22 13:35:25 +05:30
pom.xml Apple FR68: New database encryption cipher (#214) 2023-01-31 13:18:28 +01:00
requirements.txt CLOUDSTACK-8855 Improve Error Message for Host Alert State and reconnect host API. (#2387) 2018-03-14 15:27:43 -03:00

README.md

Apache CloudStack Build Status Quality Gate Status Lines of Code GitHub language count GitHub top language

Apache CloudStack

Apache CloudStack is open source software designed to deploy and manage large networks of virtual machines, as a highly available, highly scalable Infrastructure as a Service (IaaS) cloud computing platform. CloudStack is used by a number of service providers to offer public cloud services, and by many companies to provide an on-premises (private) cloud offering, or as part of a hybrid cloud solution.

CloudStack is a turnkey solution that includes the entire "stack" of features most organizations want with an IaaS cloud: compute orchestration, Network-as-a-Service, user and account management, a full and open native API, resource accounting, and a first-class User Interface (UI).

CloudStack currently supports the most popular hypervisors: VMware vSphere, KVM, XenServer, XenProject and Hyper-V as well as OVM and LXC containers.

Users can manage their cloud with an easy to use Web interface, command line tools, and/or a full-featured query based API.

For more information on Apache CloudStack, please visit the website

Who Uses CloudStack?

  • There are more than 150 known organizations using Apache CloudStack (or a commercial distribution of CloudStack). Our users include many major service providers running CloudStack to offer public cloud services, product vendors who incorporate or integrate with CloudStack in their own products, organizations who have used CloudStack to build their own private clouds, and systems integrators that offer CloudStack related services.

  • See our case studies highlighting successful deployments of Apache CloudStack.

  • See the up-to-date list of current users.

  • If you are using CloudStack in your organization and your company is not listed above, please complete our brief adoption survey. We're happy to keep your company name anonymous if you require.

Getting Started

  • Download a released version
  • Build from source with the instructions in the INSTALL.md file.

Getting Source Repository

Apache CloudStack project uses Git. The official Git repository is at:

https://gitbox.apache.org/repos/asf/cloudstack.git

And a mirror is hosted on Github:

https://github.com/apache/cloudstack

The Github mirror is strictly read only and provides convenience to users and developers to explore the code and for the community to accept contributions via Github pull requests.

Getting Involved and Contributing

Interested in helping out with Apache CloudStack? Great! We welcome participation from anybody willing to work The Apache Way and make a contribution. Note that you do not have to be a developer in order to contribute to Apache CloudStack. We need folks to help with documentation, translation, promotion etc. See our contribution page.

If you're interested in learning more or participating in the Apache CloudStack project, the mailing lists are the best way to do that. While the project has several communications channels, the mailing lists are the most active and the official channels for making decisions about the project itself.

Mailing lists:

Report and/or check bugs on Github and check our developer page for contributing code.

News and Events

Reporting Security Vulnerabilities

If you've found an issue that you believe is a security vulnerability in a released version of CloudStack, please report it to security@cloudstack.apache.org with details about the vulnerability, how it might be exploited, and any additional information that might be useful.

For more details, please visit our security page.

License

Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Please see the LICENSE file included in the root directory of the source tree for extended license details.

Notice of Cryptographic Software

This distribution includes cryptographic software. The country in which you currently reside may have restrictions on the import, possession, use, and/or re-export to another country, of encryption software. BEFORE using any encryption software, please check your country's laws, regulations and policies concerning the import, possession, or use, and re-export of encryption software, to see if this is permitted. See http://www.wassenaar.org/ for more information.

The U.S. Government Department of Commerce, Bureau of Industry and Security (BIS), has classified this software as Export Commodity Control Number (ECCN) 5D002.C.1, which includes information security software using or performing cryptographic functions with asymmetric algorithms. The form and manner of this Apache Software Foundation distribution makes it eligible for export under the License Exception ENC Technology Software Unrestricted (TSU) exception (see the BIS Export Administration Regulations, Section 740.13) for both object code and source code.

The following provides more details on the included cryptographic software:

  • CloudStack makes use of JaSypt cryptographic libraries.
  • CloudStack has a system requirement of MySQL, and uses native database encryption functionality.
  • CloudStack makes use of the Bouncy Castle general-purpose encryption library.
  • CloudStack can optionally interact with and control OpenSwan-based VPNs.
  • CloudStack has a dependency on and makes use of JSch - a java SSH2 implementation.