clustoapi.server: Server main module

Overview

The Clusto API Server will work as an alternative to the direct database access traditional clusto commands and libraries use.

Sometimes your database has restricted access or you want to expose your clusto information across facilities but don’t want to risk opening up your database port to the outside world or don’t have the required know-how to do it in a secure manner.

Exposing an HTTP(S) endpoint is a more common problem and as such there are several well-understood solutions. Scalability and security are also points you have to consider.

The Clusto API Server should thus have the following required features:

  • Complete object manipulation: create, delete, insert, and remove objects, besides displaying them.
  • Complete object attribute manipulation: add, delete, update, query, and displaying attributes
  • Resource manipulation: allocate and deallocate objects from resources
  • Querying

Custom Headers

The Clusto API Server comes with the ability to pass certain headers to multiple operations.

Clusto-Mode:Determines if an object is compact or expanded. Compaction of objects helps speed up response time for multiple object lookups. expanded is the default mode if the function returns only one object, and is compact by default for all listing functions.
Clusto-Per-Page:
 Number of entities to return when pagination is requested. Defaults to 50.
Clusto-Page:Requests the current page in a list of entities, delimited by Clusto-Per-Page.
Clusto-Pages:Response only. This header returns the total number of pages to the requester.
Clusto-Minify:If set to True (not case sensitive), clusto will not give a response that has been pretty-printed.

Configurable Response Headers

The Clusto API Server comes with the ability to define and configure static response headers within the clusto.conf.

Example:To enable CORS, a line in the clusto.conf would read: response_headers = Access-Control-Allow-Origin:*

API Docs

clustoapi.server.build_docs(module='clustoapi.server')[source]

This will build documentation for the given module and all its methods. If python-rest is available, it will attempt to parse it as a restructured text document. You can get to the docs by going to the __doc__ endpoint on each mounted application, the main __doc__ endpoint, or on the main endpoint:

$ ${get} ${server_url}/__doc__
<?xml version="1.0" encoding="utf-8" ?>
...
HTTP: 200
Content-type: text/html; charset=UTF-8

If you pass the Accept headers and specify text/plain, you should get the plain text version back

$ ${get} -H 'Accept: text/plain' ${server_url}/__doc__
...
HTTP: 200
Content-type: text/plain

$ diff -q <( curl -s -H 'Accept: text/plain' ${server_url}/__doc__ ) <( curl -s -H 'Accept: text/plain' ${server_url}/ ) && echo 'equal' || echo 'diff'
equal

In the test config, CORS is configured to be returned with every response header.

Or:

If you try to get a non-configured header, it shouldn’t be in the output

clustoapi.server.favicon()[source]

Send an HTTP code to clients so they stop asking for favicon. Example:

$ ${get} -o /dev/null ${server_url}/favicon.ico
HTTP: 410
Content-type: text/html; charset=UTF-8
clustoapi.server.get_by_attr()[source]

One of the main clusto operations. Parameters:

  • Required: the key parameter
  • Optional: the subkey parameter
  • Optional: the value parameter

Examples:

$ ${get} ${server_url}/by-attr
"Provide a key to use get_by_attr"
HTTP: 412
Content-type: application/json

$ ${get} -d 'key=nonkey' ${server_url}/by-attr
[]
HTTP: 200
Content-type: application/json

$ ${get} -d 'key=key1' -d 'subkey=subkey1' -d 'value=value1' ${server_url}/by-attr
[
    "/basicserver/testserver1"
]
HTTP: 200
Content-type: application/json

$ ${get} -H 'Clusto-Mode: expanded' -d 'key=key1' ${server_url}/by-attr
[
    {
        "attrs": [
            {
                "datatype": "string",
                "key": "key1",
                "number": null,
                "subkey": "subkey1",
                "value": "value1"
            }
        ],
        "contents": [],
        "driver": "basicserver",
        "ips": [],
        "name": "testserver1",
        "parents": [
            "/pool/singlepool",
            "/pool/multipool"
        ],
        "type": "server"
    },
    {
        "attrs": [
            {
                "datatype": "string",
                "key": "key1",
                "number": null,
                "subkey": "subkey2",
                "value": "value2"
            }
        ],
        "contents": [],
        "driver": "basicserver",
        "ips": [],
        "name": "testserver2",
        "parents": [
            "/pool/multipool"
        ],
        "type": "server"
    }
]
HTTP: 200
Content-type: application/json
clustoapi.server.get_by_name(name)[source]

One of the main clusto operations. Parameters:

  • Required path parameter: name - The name you’re looking for
  • Optional: driver - If provided, a driver check will be added to ensure the resulting object is the type you’re expecting

Note

This function returns expanded objects by default in order to reduce the amount of required custom headers. Therefore, the header is not required to receive expanded objects.

Examples:

$ ${get} ${server_url}/by-name/nonserver
"Object "nonserver" not found (nonserver does not exist.)"
HTTP: 404
Content-type: application/json

$ ${get} -H 'Clusto-Mode: compact' ${server_url}/by-name/testserver1
"/basicserver/testserver1"
HTTP: 200
Content-type: application/json

$ ${get} ${server_url}/by-name/testserver1
{
    "attrs": [
        {
            "datatype": "string",
            "key": "key1",
            "number": null,
            "subkey": "subkey1",
            "value": "value1"
        }
    ],
    "contents": [],
    "driver": "basicserver",
    "ips": [],
    "name": "testserver1",
    "parents": [
        "/pool/singlepool",
        "/pool/multipool"
    ],
    "type": "server"
}
HTTP: 200
Content-type: application/json

$ ${get} -d 'driver=pool' ${server_url}/by-name/testserver1
"The driver for object "testserver1" is not "pool""
HTTP: 409
Content-type: application/json

$ ${get} -d 'driver=nondriver' ${server_url}/by-name/testserver1
"The driver "nondriver" is not a valid driver"
HTTP: 412
Content-type: application/json
clustoapi.server.get_by_names()[source]

One of the main clusto operations. Parameters:

  • Required parameter: At least one name parameter

Returns HTTP: 404 when all entites requested do not exist and HTTP: 206 when a percent of entities requested do not exist.

Examples:

$ ${get} ${server_url}/by-names
"Provide at least one name to get data from"
HTTP: 412
Content-type: application/json

$ ${get} -d 'name=nonserver' ${server_url}/by-names
[
    null
]
HTTP: 404
Content-type: application/json

$ ${get} -d 'name=testserver1' -d 'name=nonserver' ${server_url}/by-names
[
    "/basicserver/testserver1",
    null
]
HTTP: 206
Content-type: application/json

$ ${get} -H 'Clusto-Mode: expanded' -d 'name=testserver1' -d 'name=testserver2' ${server_url}/by-names
[
    {
        "attrs": [
            {
                "datatype": "string",
                "key": "key1",
                "number": null,
                "subkey": "subkey1",
                "value": "value1"
            }
        ],
        "contents": [],
        "driver": "basicserver",
        "ips": [],
        "name": "testserver1",
        "parents": [
            "/pool/singlepool",
            "/pool/multipool"
        ],
        "type": "server"
    },
    {
        "attrs": [
            {
                "datatype": "string",
                "key": "key1",
                "number": null,
                "subkey": "subkey2",
                "value": "value2"
            }
        ],
        "contents": [],
        "driver": "basicserver",
        "ips": [],
        "name": "testserver2",
        "parents": [
        "/pool/multipool"
        ],
        "type": "server"
    }
]
HTTP: 200
Content-type: application/json

$ ${get} -d 'name=nonserver1' -d 'name=nonserver2' ${server_url}/by-names
[
    null,
    null
]
HTTP: 404
Content-type: application/json
clustoapi.server.get_driverlist()[source]

Returns clusto.driverlist, a list of drivers and their qualified class paths.

Examples:

$ ${get} ${server_url}/driverlist
{
    "basicappliance": "clusto.drivers.devices.appliance.basicappliance.BasicAppliance",
    "basiccage": "clusto.drivers.locations.datacenters.basiccage.BasicCage",
    "basicconsoleserver": "clusto.drivers.devices.consoleservers.basicconsoleserver.BasicConsoleServer",
    "basicdatacenter": "clusto.drivers.locations.datacenters.basicdatacenter.BasicDatacenter",
    "basicnetworkswitch": "clusto.drivers.devices.networkswitches.basicnetworkswitch.BasicNetworkSwitch",
    "basicpowerstrip": "clusto.drivers.devices.powerstrips.basicpowerstrip.BasicPowerStrip",
    "basicrack": "clusto.drivers.locations.racks.basicrack.BasicRack",
    "basicserver": "clusto.drivers.devices.servers.basicserver.BasicServer",
    "basicvirtualserver": "clusto.drivers.devices.servers.basicserver.BasicVirtualServer",
    "basiczone": "clusto.drivers.locations.zones.basiczone.BasicZone",
    "clustometa": "clusto.drivers.base.clustometa.ClustoMeta",
    "device": "clusto.drivers.base.device.Device",
    "entity": "clusto.drivers.base.driver.Driver",
    "exclusive_pool": "clusto.drivers.categories.pool.ExclusivePool",
    "ipmanager": "clusto.drivers.resourcemanagers.ipmanager.IPManager",
    "location": "clusto.drivers.base.location.Location",
    "pool": "clusto.drivers.categories.pool.Pool",
    "resourcemanager": "clusto.drivers.base.resourcemanager.ResourceManager",
    "simpleentitynamemanager": "clusto.drivers.resourcemanagers.simplenamemanager.SimpleEntityNameManager",
    "simplenamemanager": "clusto.drivers.resourcemanagers.simplenamemanager.SimpleNameManager",
    "unique_pool": "clusto.drivers.categories.pool.UniquePool"
}
HTTP: 200
Content-type: application/json
clustoapi.server.get_from_pools()[source]

One of the main clusto operations. Parameters:

  • Required: at least one pool parameter
  • Optional: one or more driver parameter to filter out results
  • Optional: one or more type parameter to filter out results
  • Optional: a boolean children parameter to search for children recursively (True by default)

Examples:

$ ${get} ${server_url}/from-pools
"Provide at least one pool to get data from"
HTTP: 412
Content-type: application/json

$ ${get} -H 'Clusto-Page: notanint' -d 'pool=emptypool' ${server_url}/from-pools
"invalid literal for int() with base 10: 'notanint'"
HTTP: 400
Content-type: application/json

$ ${get} -d 'pool=emptypool' ${server_url}/from-pools
[]
HTTP: 200
Content-type: application/json

$ ${get} -d 'pool=singlepool' -d 'pool=multipool' ${server_url}/from-pools
[
    "/basicserver/testserver1"
]
HTTP: 200
Content-type: application/json

$ ${get} -H 'Clusto-Mode: expanded' -d 'pool=multipool' ${server_url}/from-pools
[
    {
        "attrs": [
            {
                "datatype": "string",
                "key": "key1",
                "number": null,
                "subkey": "subkey1",
                "value": "value1"
            }
        ],
        "contents": [],
        "driver": "basicserver",
        "ips": [],
        "name": "testserver1",
        "parents": [
            "/pool/singlepool",
            "/pool/multipool"
        ],
        "type": "server"
    },
    {
        "attrs": [
            {
                "datatype": "string",
                "key": "key1",
                "number": null,
                "subkey": "subkey2",
                "value": "value2"
            }
        ],
        "contents": [],
        "driver": "basicserver",
        "ips": [],
        "name": "testserver2",
        "parents": [
            "/pool/multipool"
        ],
        "type": "server"
    }
]
HTTP: 200
Content-type: application/json

$ ${get} -H 'Clusto-Page: 1'  -H 'Clusto-Per-Page: 1' -d 'pool=multipool' ${server_url}/from-pools
[
    "/basicserver/testserver1"
]
HTTP: 200
Content-type: application/json

$ ${get} -H 'Clusto-Page: 1'  -H 'Clusto-Per-Page: 100' -d 'pool=multipool' ${server_url}/from-pools
[
    "/basicserver/testserver1",
    "/basicserver/testserver2"
]
HTTP: 200
Content-type: application/json

$ ${get} -H 'Clusto-Page: 100'  -H 'Clusto-Per-Page: 100' -d 'pool=multipool' ${server_url}/from-pools
[]
HTTP: 200
Content-type: application/json

$ ${get} -H 'Clusto-Minify: True' -d 'pool=multipool' ${server_url}/from-pools
["/basicserver/testserver1", "/basicserver/testserver2"]
HTTP: 200
Content-type: application/json
clustoapi.server.get_typelist()[source]

Returns clusto.typelist, a list of the most recent drivers to register with that type.

Examples:

$ ${get} ${server_url}/typelist
{
    "appliance": "clusto.drivers.devices.appliance.basicappliance.BasicAppliance",
    "cage": "clusto.drivers.locations.datacenters.basiccage.BasicCage",
    "clustometa": "clusto.drivers.base.clustometa.ClustoMeta",
    "consoleserver": "clusto.drivers.devices.consoleservers.basicconsoleserver.BasicConsoleServer",
    "datacenter": "clusto.drivers.locations.datacenters.basicdatacenter.BasicDatacenter",
    "generic": "clusto.drivers.base.device.Device",
    "location": "clusto.drivers.base.location.Location",
    "networkswitch": "clusto.drivers.devices.networkswitches.basicnetworkswitch.BasicNetworkSwitch",
    "pool": "clusto.drivers.categories.pool.UniquePool",
    "powerstrip": "clusto.drivers.devices.powerstrips.basicpowerstrip.BasicPowerStrip",
    "rack": "clusto.drivers.locations.racks.basicrack.BasicRack",
    "resourcemanager": "clusto.drivers.resourcemanagers.ipmanager.IPManager",
    "server": "clusto.drivers.devices.servers.basicserver.BasicVirtualServer",
    "zone": "clusto.drivers.locations.zones.basiczone.BasicZone"
}
HTTP: 200
Content-type: application/json
clustoapi.server.main()[source]

Main entry point for the clusto-apiserver console program

clustoapi.server.meta()[source]

This call just returns a mapping of all currently installed applications.

$ ${get} ${server_url}/__meta__
...
HTTP: 200
Content-type: application/json
clustoapi.server.options(**kwargs)[source]

Defined from w3.org:

“The OPTIONS method represents a request for information about the communication options available on the request/response chain identified by the Request-URI. This method allows the client to determine the options and/or requirements associated with a resource, or the capabilities of a server, without implying a resource action or initiating a resource retrieval.”

The clusto-apiserver team plans to roll this out to individual resources once it has been used a proper amount, but for now we will return OPTIONS with the minimum amount of required headers (and an empty content) no matter what resource is requested.

$ ${head} -X OPTIONS ${server_url}/
HTTP/1.0 204 No Content
...
Content-Length: 0

$ ${head} -X OPTIONS ${server_url}/return/headers/no/matter/where
HTTP/1.0 204 No Content
...
Content-Length: 0

The same applies to any mounted application:

$ ${head} -X OPTIONS ${server_url}/entity/
HTTP/1.0 204 No Content
...
Content-Length: 0

$ ${head} -X OPTIONS ${server_url}/attribute/who/knows/where
HTTP/1.0 204 No Content
...
Content-Length: 0
clustoapi.server.version()[source]

This shows the current version running, example

$ ${get} ${server_url}/__version__
"${server_version}"
HTTP: 200
Content-type: application/json

If you make a HEAD request to the / endpoint, the response is also the version string, as that’s less heavy to build than the regular / page:

$ ${head} ${server_url}/
HTTP/1.0 200 OK
...

clustoapi.util: Miscellaneous util module

clustoapi.util.dumps(obj, code=200, headers={})[source]

Dumps a given object as a JSON string in an HTTP Response object. Will circumvent pretty-printing if Clusto-Minify header is True.

clustoapi.util.get(name, driver=None)[source]

Tries to fetch a clusto object from a given name, optionally validating the driver given. Returns:

  • HTTP Error 404 if the object could not be found
  • HTTP Error 409 if the object does not match the expected driver
  • Clusto object otherwise
clustoapi.util.page(ents, current=1, per=50)[source]

Takes a list of entities and drops all from a list but the current page. Returns a tuple that has the entities and also a page total so it may be returned to the client.

clustoapi.util.show(obj, mode='')[source]

Will return the expanded or compact representation of a given object

clustoapi.util.typecast(value, datatype, mask='%Y-%m-%dT%H:%M:%S.%f')[source]

Takes a string and a valid clusto datatype and attempts to cast the value to the specified datatype. Will error out if a ValueError is incurred or a relation does not exist. Will aslo take a strptime format as mask because typcasting datetimes is hard.

clustoapi.util.unclusto(obj)[source]

Convert an object to a representation that can be safely serialized into JSON.