knife-cloudstack¶ ↑

<img src=“https://codeclimate.com/github/CloudStack-extras/knife-cloudstack.png” /> <img src=“https://gemnasium.com/CloudStack-extras/knife-cloudstack.png” alt=“Dependency Status” />

This is the Edmunds Knife plugin for CloudStack. This plugin gives Knife the ability to create, bootstrap and manage CloudStack instances.

This plugin is distributed as a Ruby Gem. To install it, run:

gem install knife-cloudstack

Depending on your system’s configuration, you may need to run this command with root privileges.

In order to communicate with the CloudStack API you will have to tell Knife about your CloudStack API Key, your Secret Key and the URL of the API endpoint. The easiest way to accomplish this is to create some entries in your knife.rb file:

knife[:cloudstack_url] = "http://yourcloudstackserver.com:8080/client/api
knife[:cloudstack_api_key]  = "Your CloudStack API Key"
knife[:cloudstack_secret_key] = "Your CloudStack Secret Key"

If your knife.rb file will be checked into a SCM system (ie readable by others) you may want to read the values from environment variables:

knife[:cloudstack_url] = "#{ENV['CLOUDSTACK_URL']}"
knife[:cloudstack_api_key] = "#{ENV['CLOUDSTACK_API_KEY']}"
knife[:cloudstack_secret_key] = "#{ENV['CLOUDSTACK_SECRET_KEY']}"

You also have the option of passing your CloudStack URL and API/Secret Keys into the individual knife subcommands using the -U (or --cloudstack-url), -A (or --cloudstack-api-key ) -K (or --cloudstack-secret-key) command options

# provision a new small RHEL 5.6 webserver
knife cs server create my-server -r 'role[webserver]' -S "small" -T "RHEL 5.6 Base" -A 'Your CloudStack API Key' -K "Your CloudStack Secret Key" --distro "rhel5-gems"

Additionally the following options may be set in your knife.rb:

• knife

• knife

• knife

• knife

• knife

• knife

• knife

To get this plugin to work in public clouds, it is essential that the virtual network (and router) be allocated to the account. Cloudstack clouds automatically creates a virtual network when the first VM is requested to be created. Hence, it is essential to create the first VM (of a newly created account) manually(which can be terminated immediately if not required) to ensure the virtual network is created.

This plugin provides the following Knife subcommands. Specific command options can be found by invoking the subcommand with a --help flag

--filter Filters your output on the fields that you specify. Filter can be a string ‘name’ or regexp ‘/name/i’

Example: knife cs server list --filter "instancename:/i-xxx/i, account:accname"

--fields The fields that you want to display in your output. Default outputs will be shown if no fields are defined.

Example: knife cs server list --fields "name, instancename, domain"

--fieldlist This will display all information returned by the cloudstack API. These fields can be used within the --fields or --filter to create custom output. The data displayed is based on first result returned in your output. You can get other result by using the --filter option.

--noheader this will remove the column header from the output.

Provisions a new server in CloudStack and then performs a Chef bootstrap (using the SSH protocol). The goal of the bootstrap is to get Chef installed on the target system so it can run Chef Client with a Chef Server. The main assumption is a baseline OS installation exists (provided by the provisioning). It is primarily intended for Chef Client systems that talk to a Chef server. By default the server is bootstrapped using the ‘chef-full’ template (default bootstrap option for knife, (Ref. docs.opscode.com/knife_bootstrap.html). This can be overridden using the -d or --template-file command options. VM provided with --no-bootstrap option have no forwarded ports or forwared ip rules (in case --static-nat is used).

By default, new servers are allocated a public IP address mapping to the CloudStack private IP address. If you do not want this behavior, pass the --no-public-ip option.

Forcing cloustack to reserve and use a certain private ipv4 address when deploying an instance inside an advanced zone is possible by passing <option>–private-ip</option> followed by the intended ipv4 address. Ip Address must be contained within the network mask of the isolated network, of course.

Example: knife cs server create testvm --private-ip 192.168.1.1 --networks mynetwork

Here mynetwork must have a network mask for example like 192.168.1.0/24.

The --port-rules option takes a comma separated list of port forwarding rules. These rules are created on the virtual public ip address of the server. Note that this option only applies to servers using a virtual network; it is ignored if the server’s public ip address is on a direct attached network.

Port forwarding rules have the syntax PUBLIC_PORT[:PRIVATE_PORT[:PROTOCOL]]. PRIVATE_PORT and PROTOCOL are optional. The default value of PRIVATE_PORT is PUBLIC_PORT and the default PROTOCOL is ‘TCP’. For example, a rule to forward from public port 80 to private port 7000 would look like 80:7000:TCP. Since ‘TCP’ is the default protocol, the rule can be shortened to 80:7000. A rule can even be shortened to a single number when the public and private ports are the same. For example, a rule to forward from public port 25 to private port 25 can be stated as simply 25. A list of such rules for a webserver might look like 80,443.

The --ipfwd-rules option takes a comma separated list of ip forwarding rules. These rules are created on public ip appdress assigned obtained with --static-nat option. (Ref. cloudstack.apache.org/docs/api/apidocs-4.0.0/root_admin/createIpForwardingRule.html)

Ip forwarding rules have the syntax START_PORT[:END_PORT[:PROTOCOL]]. END_PORT and PROTOCOL are optional. The default value of END_PORT is START_PORT and the default PROTOCOL is ‘TCP’. For example, a rule to forward ports range from 1024 to 10000 would look like 1024:10000:TCP. Since ‘TCP’ is the default protocol, the rule can be shortened to 1024:10000. A rule can even be shortened to a single number when the start and end ports are the same. For example, a rule to forward port 22 can be stated as simply 22. A list of such rules for a webserver might look like 80,443.

The -f, --fw-rules option takes a comma separated list of firewall rules which are applied to the public ip address assigned to the current server.

Firewall rules have the syntax PROTOCOL[:CIDR_LIST[:START_PORT[:END_PORT]]]. START_PORT and END_PORT must not be specified when PROTOCOL is ICMP, CIDR_LIST is always optional. The default value of END_PORT is START_PORT, the default CIDR_LIST is ‘0.0.0.0/0’. For example, a rule to open firewall for port 80 to everyone would look like TCP::80 and a rule to open ICMP to internal network would look like ICMP:10.0.0.0/8.

Deletes an existing server in the currently configured CloudStack account. PLEASE NOTE - this does not delete the associated node and client objects from the Chef server.

Displays a list of all servers in the currently configured CloudStack account. PLEASE NOTE - this shows all servers associated with the cloudstack account including servers that are not registered with a Chef server.

--listall This will list all the servers, depending on the account that you are using.

--keyword Filters your output on the instance name that you specify:

Example: --keyword "i-324", This will display all servers with <tt>'i-324'</tt> inside the instancename.

--name Filters your output on the name that you specify.

Example: --name "webserver", This will display all servers that contains 'webserver' inside the hostname.

--action This action will be executed on the output of the list command. Actions can be start, stop or destroy.

Displays a list of all networks available in the currently configured CloudStack account. A network can be specified when creating a new server by passing the network name as an argument to the -W (or –network) option of the knife cs server create command.

--listall This will list all the networks, depending on the account that you are using.

--keyword Filters your output on the keyword that you specify.

Example: --keyword "network1", This will display all networks with 'network1' inside the name.

Displays a list of all service offerings available in the currently configured CloudStack account. A service offering can be specified when creating a new server by passing the name as an argument to the -S (or –service) option of the knife cs server create command.

--keyword Filters your output on the service name, just like --name here below.

--name Filters your output on the name that you specify.

Example: --name "medium" or --keyword "medium", This will display all cpu service offerings with 'medium' inside the name.

Displays a list of all templates in the currently configured CloudStack account. Featured templates are displayed by default. Use the -F (or –filter) option to use a different filter. The allowable filter values are:

• featured - templates that are featured and are public (this is the default)

• self - templates that have been registered/created by the owner

• self-executable - templates that have been registered/created by the owner that can be used to deploy a new VM

• executable - all templates that can be used to deploy a new VM

• community - templates that are public

A template can be specified when creating a new server by passing the template name as an argument to the -T (or –template) option of the knife cs server create command.

--listall This will list all the templates, depending on the account that you are using.

Creates a template based on a volumeID

Returns a link where an extractable template can be downloaded

Creates a template based on a file

Creates a volume based on a name

Displays a list of all zones available in the currently configured CloudStack account. A zone can be specified when creating a new server by passing the zone name as an argument to the -Z (or –zone) option of the knife cs server create command.

--keyword Filters your output on the keyword that you specify.

Example: --keyword "zone1", This will display all zones with 'zone1' inside the name

Convenience command that displays the public ip address and fqdn for all servers. Matches /etc/hosts file format.

--listall This will list all the servers, depending on the account that you are using.

--keyword Filters your result based on a keyword.

--name Filters your output on the name that you specify.

Displays all accounts that are currently in your cloudstack environment.

--listall This will list all the possible results, depending on the account that you are using.

--keyword Filters your result based on a keyword.

--name Filters your output on the name that you specify.

Displays all clusters that are currently available in your cloudstack environment.

--keyword Filters your result based on a keyword.

--name Filters your output on the name that you specify.

Displays all the configuration settings that are configured inside the cloudstack environment.

--keyword Filters your result based on a keyword.

--name Filters your output on the name that you specify.

Displays all the disks that are available within cloudstack.

--keyword Filters your result based on a keyword.

--name Filters your output on the name that you specify.

Displays all domains within the cloudstack environment.

--listall This will list all the domains, depending on the account that you are using.

Displays all firewall rules that are currently active within your cloudstack environment.

--listall This will list all the firewallrules, depending on the account that you are using.

--keyword Filters your result based on a keyword.

Displays all hosts from the cloudstack environment.

--name Filters your output on the name that you specify.

Displays all iso’s that are available within the cloudstack environment.

--listall This will list all the iso’s, depending on the account that you are using.

--keyword Filters your result based on a keyword.

--name Filters your output on the name that you specify.

Displays all the os types that are available within the cloudstack environment.

--keyword Filters your output on the keyword that you specify.

Displays all os categories that are available --keyword Filters your output on the keyword that you specify.

Displays all the pods that are currently available within the cloudstack environment.

--keyword Filters your result based on a keyword.

--name Filters your output on the name that you specify.

Displays all the projects that are within the cloudstack environment.

--listall This will list all the projects, depending on the account that you are using.

--keyword Filters your result based on a keyword.

--name Filters your output on the name that you specify.

Displays all public ip’s that are used within the cloudstack environment.

--listall This will list all the public ip’s, depending on the account that you are using.

Displays all the routers that are within the cloudstack environment.

--listall This will list all the routers, depending on the account that you are using.

--keyword Filters your result based on a keyword.

--name Filters your output on the name that you specify.

Displays all the users that are available within your account.

--listall This will list all the users, depending on the account that you are using.

--keyword Filters your output on the keyword that you specify.

Displays all the volumes that are currently available within the cloudstack environment.

--listall This will list all the volumes, depending on the account that you are using.

--keyword Filters your output on the keyword that you specify.

--name Filters your output on the name that you specify.

Creates a “stack” of servers based on a JSON definition file. Simple orchestration can be performed by specifying one or more actions to be executed after a server (or group of servers) is created.

--skip-existing Skip erroring on any servers already created in the stack (default is false)

{
  "name": "hadoop_cluster_a",
  "description": "A small hadoop cluster with hbase",
  "version": "1.0",
  "environment": "production",
  "servers": [
    {
      "name": "zookeeper-a, zookeeper-b, zookeeper-c",
      "description": "Zookeeper nodes",
      "template": "rhel-5.6-base",
      "service": "small",
      "port_rules": "2181",
      "run_list": "role[cluster_a], role[zookeeper_server]",
      "actions": [
        { "knife_ssh": ["role:zookeeper_server", "sudo chef-client"] }
      ]
    },
    {
      "name": "hadoop-master",
      "description": "Hadoop master node",
      "template": "rhel-5.6-base",
      "service": "large",
      "networks": "app-net, storage-net",
      "port_rules": "50070, 50030, 60010",
      "run_list": "role[cluster_a], role[hadoop_master], role[hbase_master]"
    },
    {
      "name": "hadoop-worker-a hadoop-worker-b hadoop-worker-c",
      "description": "Hadoop worker nodes",
      "template": "rhel-5.6-base",
      "service": "medium",
      "port_rules": "50075, 50060, 60030",
      "run_list": "role[cluster_a], role[hadoop_worker], role[hbase_regionserver]",
      "actions": [
        { "knife_ssh": ["role:hadoop_master", "sudo chef-client"] },
        { "http_request": "http://${hadoop-master}:50070/index.jsp" }
      ]
    }
  ]
}

name

Stack name. May be used in the future to e.g. look up a stack in a databag.

version

Stack version. This attribute is not currently used.

environment

Chef environment used for all servers in the stack.

servers

List of servers to create.

name

The name of the new server. Specify a list of names separated by commas and/or spaces to create multiple servers.

description

Server description.

template

Cloudstack template name.

service

Cloudstack service offering name.

port_rules

Port forwarding rules for virtual networks. These are ignored when using direct or other network types.

run_list

Chef run list.

actions

List of actions to perform after the server (or group of servers) is created.

knife-ssh

Performs a knife-ssh command. Takes the search query and the command to execute as an array of arguments.

http_request

Performs an http request. Supports references to server names in the stack with ${server_name}.

The following actions are under development:

runlist_add

Adds an entry from the run list.

runlist_remove

Removes an entry from the run list.

Deletes all servers in the specified stack definition.

Starts the specified virtual machine(s).

Stops the specified virtual machine(s). Force stop is optional.

Reboots the specified virtual machines(s).

Author

Ryan Holmes <rholmes@edmunds.com>

Author

KC Braunschweig <kcbraunschweig@gmail.com>

Author

John E. Vincent <lusis.org+github.com@gmail.com>

Author

Sander Botman <sbotman@schubergphilis.com>

Author

Frank Breedijk <fbreedijk@schubergphilis.com>

Author

Jeremy Baumont <jbaumont@schubergphilis.com>

Copyright

Copyright © 2011 Edmunds, Inc.

License

Apache License, Version 2.0

Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

knife is a trademark of Opscode

CloudStack is a trademark or registered trademark of The Apache Software Foundation

The use of these marks is nominative in describing the function and integration of this software and does not imply endorsement by Opscode, the Apache Software Foundation, or Apache CloudStack.