About TOSCA

The TOSCA acronym stands for Topology and Orchestration Specification for Cloud Applications. It’s an OASIS standard.

The purpose of the TOSCA project is to represent an application by its topology and formalize it using the TOSCA grammar.

The [TOSCA-Simple-Profile-YAML-v1.0] current specification in YAML introduces the following concepts.

• TOSCA YAML service template: A YAML document artifact containing a (TOSCA) service template that represents a Cloud application.
• TOSCA processor: An engine or tool that is capable of parsing and interpreting a TOSCA YAML service template for a particular purpose. For example, the purpose could be validation, translation or visual rendering.
• TOSCA orchestrator (also called orchestration engine): A TOSCA processor that interprets a TOSCA YAML service template then instantiates and deploys the described application in a Cloud.
• TOSCA generator: A tool that generates a TOSCA YAML service template. An example of generator is a modeling tool capable of generating or editing a TOSCA YAML service template (often such a tool would also be a TOSCA processor).
• TOSCA archive (or TOSCA Cloud Service Archive, or “CSAR”): a package artifact that contains a TOSCA YAML service template and other artifacts usable by a TOSCA orchestrator to deploy an application.

My work with TOSCA

I do believe that TOSCA may be a very good leverage to port a “legacy application” (aka born in the datacenter application) into a cloud ready application without rewriting it completely to be cloud compliant. To be clear, It may act on the hosting and execution plan of the application, and not on the application itself.

A single wordpress installation in a TOSCA way as written here is represented like that

While I was learnig GO, I have developped a TOSCA lib and a TOSCA processor which are, by far, not idiomatic GO.

Here are two screenshots of the rendering in a web page made with my tool (and the graphviz product):

The graph representation of a lifecycle of Single instance wordpress

The TOSCA file is parsed with the help of the TOSCALIB and then it fills an adjacency matrix (see FillAdjacencyMatrix)

The graphviz take care of the (di)graph representation.

What I would like to do now, is a little bit more: I would like to play with the graph and query it Then I should perform requests on this graph. For example I could ask:

• What are the steps to go from the state Initial of the application, to the state running ?
• What are the steps to go from stop to delete
• …

and that would be the premise of a TOSCA orchestrator.

The digraph go code

I’ve recently discoverd the digraph tool, that I will use for querying the graphs. The digraph is represented as a map with a node as a key and its immediates successors as values:

1// A graph maps nodes to the non-nil set of their immediate successors.
2type graph map[string]nodeset
3
4type nodeset map[string]bool

From TOSCA to digraph

What I must do is to parse the adjacency matrix, get the “lifecycle action” related to the id and fill the graph g.

Let’s go

Considering the digraph code, what I need to do is simply to override the parse method.

Principle

I will fill the graph with a string composed of nodename:action as key. For example, if I need to do a “Configure” action of node “A” after a “Start” action on node “B”, I will have the following entry in the map:

1g["B:Start"] = "A:Configure"

So What I need to do is to parse the adjjacency matrix, do a matching with the row id and the “node:action” name, and fill the graph g with the matching of the corresponding “node:action”.

I will fill a map with the id of the node:action as key and the corresponding label as values:

for node, template := range toscaTemplate.TopologyTemplate.NodeTemplates {
        ids[template.GetConfigureIndex()] = fmt.Sprintf("%v:Configure", node)
        ids[template.GetCreateIndex()] = fmt.Sprintf("%v:Create", node)
        ids[template.GetDeleteIndex()] = fmt.Sprintf("%v:Delete", node)
        ids[template.GetInitialIndex()] = fmt.Sprintf("%v:Initial", node)
        ids[template.GetPostConfigureSourceIndex()] = fmt.Sprintf("%v:PostConfigureSource", node)
        ids[template.GetPostConfigureTargetIndex()] = fmt.Sprintf("%v:PostconfigureTarget", node)
        ids[template.GetPreConfigureSourceIndex()] = fmt.Sprintf("%v:PreConfigureSource", node)
        ids[template.GetPreConfigureTargetIndex()] = fmt.Sprintf("%v:PreConfigureTarget", node)
        ids[template.GetStartIndex()] = fmt.Sprintf("%v:Start", node)
        ids[template.GetStopIndex()] = fmt.Sprintf("%v:Stop", node)
}

Then I can easily fill the graph g from the adjacency matrix:

row, col := toscaTemplate.AdjacencyMatrix.Dims()
        for r := 1; r < row; r++ {
                for c := 1; c < col; c++ {
                        if adjacencyMatrix.At(r, c) == 1 {
                                g.addEdges(ids[r], ids[c])
                        }
                }
        }

That’s it

The final function

Here is the final parse function

 1func parse(rd io.Reader) (graph, error) {
 2        g := make(graph)
 3        // Parse the input graph.
 4        var toscaTemplate toscalib.ToscaDefinition
 5        err := toscaTemplate.Parse(rd)
 6        if err != nil {
 7                return nil, err
 8        }
 9        // a map containing the ID and the corresponding action
10        ids := make(map[int]string)
11        // Fill in the graph with the toscaTemplate via the adjacency matrix
12        for node, template := range toscaTemplate.TopologyTemplate.NodeTemplates {
13                // Find the edges of the current node and add them to the graph
14
15                ids[template.GetConfigureIndex()] = fmt.Sprintf("%v:Configure", node)
16                ids[template.GetCreateIndex()] = fmt.Sprintf("%v:Create", node)
17                ids[template.GetDeleteIndex()] = fmt.Sprintf("%v:Delete", node)
18                ids[template.GetInitialIndex()] = fmt.Sprintf("%v:Initial", node)
19                ids[template.GetPostConfigureSourceIndex()] = fmt.Sprintf("%v:PostConfigureSource", node)
20                ids[template.GetPostConfigureTargetIndex()] = fmt.Sprintf("%v:PostconfigureTarget", node)
21                ids[template.GetPreConfigureSourceIndex()] = fmt.Sprintf("%v:PreConfigureSource", node)
22                ids[template.GetPreConfigureTargetIndex()] = fmt.Sprintf("%v:PreConfigureTarget", node)
23                ids[template.GetStartIndex()] = fmt.Sprintf("%v:Start", node)
24                ids[template.GetStopIndex()] = fmt.Sprintf("%v:Stop", node)
25        }
26
27        row, col := toscaTemplate.AdjacencyMatrix.Dims()
28        for r := 1; r < row; r++ {
29                for c := 1; c < col; c++ {
30                        if adjacencyMatrix.At(r, c) == 1 {
31                                g.addEdges(ids[r], ids[c])
32                        }
33                }
34        }
35        return g, nil
36}

Grab the source and compile it

I have a github repo with the source. It is go-gettable

go get github.com/owulveryck/digraph
cd $GOPATH/src/github.com/owulveryck/digraph && go build

EDIT As I continue to work on this tool, I have created a “blog” branch in the github which holds the version related to this post

Example

I will use the the same example as described below: the single instance wordpress.

I’ve extracted the YAML and placed in in the file tosca_single_instance_wordpress.yaml.

Let’s query the nodes first:

 1curl -s https://raw.githubusercontent.com/owulveryck/toscaviewer/master/examples/tosca_single_instance_wordpress.yaml | ./digraph nodes
 2mysql_database:Configure
 3mysql_database:Create
 4mysql_database:Start
 5mysql_dbms:Configure
 6mysql_dbms:Create
 7mysql_dbms:Start
 8server:Configure
 9server:Create
10server:Start
11webserver:Configure
12webserver:Create
13webserver:Start
14wordpress:Configure
15wordpress:Create
16wordpress:Start

so far, so good…

Now, I can I go from a Server:Create to a running instance wordpress:Start

curl -s https://raw.githubusercontent.com/owulveryck/toscaviewer/master/examples/tosca_single_instance_wordpress.yaml | ./digraph somepath server:Create wordpress:Start
server:Create
server:Configure
server:Start
mysql_dbms:Create
mysql_dbms:Configure
mysql_dbms:Start
mysql_database:Create
mysql_database:Configure
mysql_database:Start
wordpress:Create
wordpress:Configure
wordpress:Start

Cool!

Conclusion

The tool sounds ok. What I may add:

• a command to display the full lifecycle (finding the entry and the exit points in the matrix and call somepath with it)
• get the tosca artifacts and display them instead of the label to generate a deployment plan
• execute the command in goroutines to make them concurrent

And of course validate any other TOSCA definition to go through a bug hunting party