This introduces a new certificate authority framework that allows
pluggable CA provider implementations to handle certificate operations
around issuance, revocation and propagation. The framework injects
itself to `NioServer` to handle agent connections securely. The
framework adds assumptions in `NioClient` that a keystore if available
with known name `cloud.jks` will be used for SSL negotiations and
handshake.
This includes a default 'root' CA provider plugin which creates its own
self-signed root certificate authority on first run and uses it for
issuance and provisioning of certificate to CloudStack agents such as
the KVM, CPVM and SSVM agents and also for the management server for
peer clustering.
Additional changes and notes:
- Comma separate list of management server IPs can be set to the 'host'
global setting. Newly provisioned agents (KVM/CPVM/SSVM etc) will get
radomized comma separated list to which they will attempt connection
or reconnection in provided order. This removes need of a TCP LB on
port 8250 (default) of the management server(s).
- All fresh deployment will enforce two-way SSL authentication where
connecting agents will be required to present certificates issued
by the 'root' CA plugin.
- Existing environment on upgrade will continue to use one-way SSL
authentication and connecting agents will not be required to present
certificates.
- A script `keystore-setup` is responsible for initial keystore setup
and CSR generation on the agent/hosts.
- A script `keystore-cert-import` is responsible for import provided
certificate payload to the java keystore file.
- Agent security (keystore, certificates etc) are setup initially using
SSH, and later provisioning is handled via an existing agent connection
using command-answers. The supported clients and agents are limited to
CPVM, SSVM, and KVM agents, and clustered management server (peering).
- Certificate revocation does not revoke an existing agent-mgmt server
connection, however rejects a revoked certificate used during SSL
handshake.
- Older `cloudstackmanagement.keystore` is deprecated and will no longer
be used by mgmt server(s) for SSL negotiations and handshake. New
keystores will be named `cloud.jks`, any additional SSL certificates
should not be imported in it for use with tomcat etc. The `cloud.jks`
keystore is stricly used for agent-server communications.
- Management server keystore are validated and renewed on start up only,
the validity of them are same as the CA certificates.
New APIs:
- listCaProviders: lists all available CA provider plugins
- listCaCertificate: lists the CA certificate(s)
- issueCertificate: issues X509 client certificate with/without a CSR
- provisionCertificate: provisions certificate to a host
- revokeCertificate: revokes a client certificate using its serial
Global settings for the CA framework:
- ca.framework.provider.plugin: The configured CA provider plugin
- ca.framework.cert.keysize: The key size for certificate generation
- ca.framework.cert.signature.algorithm: The certificate signature algorithm
- ca.framework.cert.validity.period: Certificate validity in days
- ca.framework.cert.automatic.renewal: Certificate auto-renewal setting
- ca.framework.background.task.delay: CA background task delay/interval
- ca.framework.cert.expiry.alert.period: Days to check and alert expiring certificates
Global settings for the default 'root' CA provider:
- ca.plugin.root.private.key: (hidden/encrypted) CA private key
- ca.plugin.root.public.key: (hidden/encrypted) CA public key
- ca.plugin.root.ca.certificate: (hidden/encrypted) CA certificate
- ca.plugin.root.issuer.dn: The CA issue distinguished name
- ca.plugin.root.auth.strictness: Are clients required to present certificates
- ca.plugin.root.allow.expired.cert: Are clients with expired certificates allowed
UI changes:
- Button to download/save the CA certificates.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Reduces SSL handshake timeout to 15s, previously this was only 10s in
commit debfcdef78
- Adds an aggresive explicit wakeup to save the Nio main IO loop/handler from
getting blocked
- Fix NioTest to fail/succeed in about 60s, previously this was 300s
- Due to aggresive wakeup usage, NioTest should complete in less than 5s on most
systems. On virtualized environment this may slightly increase due to thread,
CPU burst/scheduling delays.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Uses non-blocking SSL handshake and non-blocking connections
- Uses 60s as timeout for both client/server to guard against indefinitely
blocking clients
- Unit test to prove fix, client and malicious clients trying to connect to server
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Changes;
- Upgrades maven-war plugin to 4.5 (faster war packaging)
- Upgrade spring framework to latest minor release
- Upgrade ehcache, jasypt, httpclient, httpcore and other core dependencies
- Upgrade to latest ipv6 library, fix unit test NetUtilsTest
- httpcore and httpclient are sharing same version variable
- commons-httpclient is different that httpclient, the fix gives it a separate var
- Apidocs failed to generate and get stuck with new reflections version, for now
we will continue using 0.9.8
Newer dependencies can be listed using:
mvn versions:display-dependency-updates -Dnoredist -Dsimulator -P developer,systemvm
Testing;
- Tested using Maven 3.2.1
- Local noredist build with unit tests succeeds
- CloudStack mgmt server started, basic business layer tests work
- Observed 10-15% build time improvement using new maven-war plugin
Branch: bugfix/4.5-8011 (commits are squashed in favour of a linear history)
Pull request:
https://github.com/apache/cloudstack/pull/50
This closes#50
TravisCI build summary:
https://travis-ci.org/shapeblue/cloudstack/builds/42902172
- Build passes with unit tests
- Apidocs generates successfully
- Most integration tests pass, some fail due to timeout errors, second re-run
passes some of them
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit fac7bfc5d5)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Conflicts:
pom.xml
- Refactor tests:
- Upgrade tests to use jUnit4
- Add hamcrest dependency (contribution of by Laszio Hornyak)
- Break big tests in small unit tests
- Replace assertTrue/False with complex conditions by assertThat with
specific matchers
- Remove dead code:
- Private static method never called locally
- Add test for method that validates cidrs
Signed-off-by: Laszlo Hornyak <laszlo.hornyak@gmail.com>
- StringBuffer replaced with StringBuilder
- nullcheck of tokens array removed since String.split does not return null
- unit test added
Signed-off-by: Laszlo Hornyak <laszlo.hornyak@gmail.com>
- new utility method introduced in PropertiesUtil to load properties objects from files
- RegionManagerImpl modified to use the utility method
- Tests added for both
Signed-off-by: Laszlo Hornyak <laszlo.hornyak@gmail.com>
Removed the assertion on the number of fields from ReflectUtil because it did not work with bytecode manipulation like what sonar works with.
Signed-off-by: Laszlo Hornyak <laszlo.hornyak@gmail.com>
- code comment turned to javadoc
- added braces to if and for statements to make it look more like the rest
- tests added
Signed-off-by: Laszlo Hornyak <laszlo.hornyak@gmail.com>
- javadoc changed - the old one was copy-pasted from AgentShell
- start and stop method removed - they did the same as the overridden methods
- _counter removed as it was only written, but never read
- remove from _asleep map was moved to a finally block, to make sure it is removed even in case of the thread gets interrupted
- Tests created for the above scenarios.
Signed-off-by: Laszlo Hornyak <laszlo.hornyak@gmail.com>
- removed methods that were not used
- parseLong, parseInt and parseFloat replaced with the commons-lang NumberUtils call
- Test for the remaining methods
Signed-off-by: Laszlo Hornyak <laszlo.hornyak@gmail.com>
Signed-off-by: Hugo Trippaers <htrippaers@schubergphilis.com>
Rohit Yadav
Rohit Yadav
Boris Schrijver
amoghvk
Sheng Yang
David Nalley
Frank Zhang
Daan Hoogland
Daan Hoogland
Damodar Reddy
Laszlo Hornyak
miguelaferreira
Antonio Fornie
Hugo Trippaers
wrodrigues
Jayapal
Saksham Srivastava
Min Chen
Alex Huang
Darren Shepherd
Marcus Sorensen