“What I know about writing high quality OO flows”… Lessons from a Flow Author
Document version 1 – 2013-07-19
David Akers
HPIT OO Practice Lead
Contents
Purpose of this document and the background on the content. 3
Style is a lot more than how a flow looks 4
Purpose of this document and the background on the content.
Writing software is a discipline dating back nearly 100 years. The corpus of knowledge describing “how to write great programs” is vast and it is not my intent to replace any of those works. This is an effort very specific to HP’s Operations Orchestration language and attempts to capture some good practices learned through some difficult experiences.
I have written some (bad) flows and some fairly good flows. I have reviewed a number of OO flows and find some easier to understand than others.
Many of the topics in this document come out of the HPIT effort to build a community of flow developers that work together to save money for our company.
Hopefully this is helpful to those desiring to learn to write OO flows.
Style is a lot more than how a flow looks
When I mention style, many people assume this is about mere “prettiness”. In some cases, people have readily dismissed these ideas on the grounds that we don’t have time to make things look nice, we have to get the flow finished “yesterday”. Generally, when I think about style I am thinking about reliability, debug-ability, supportability and the very important and much-dreaded “the difficult day after my flow ran perfectly for six months and now it does not”.
I was once in a performance organization and the leader used to say “I am going to take some time to save lots of time”. It was such an important insight. It is easy to get wrapped up in haste and forget about the “difficult day” that will likely find us someday.
A common lifecycle of a flow might go like this….
· You need to learn how to use a new operation or connect to a new database. You hard-code some stuff and assemble a quick flow. You forget about style, and just try to make something work. You get something to work and feel a bit accomplished. Perhaps you hit your head against the wall for a few hours trying to get just the right syntax of a wildcard for a remote command to do just what you need it to.
· You mention your discovery to someone and your Manager asks “how quickly can you get that into production”. You begin to focus on ”getting the flow ready” and testing it all out. Since the flow wasn’t written for prime time, you do your best to go back and make it good enough.
· You do testing and find a couple of unique situations and find clever solutions – “Adding a test here..” and “using a flow variable a different way there…”. Before long you have the flow working fairly well with the available test data.
· The flow goes to Production and starts running. You watch it for a while and are pleased that it does whatever it is supposed to do. You are congratulated and asked to work on the next opportunity.
· Five months later, you get an email from someone you have never met asking why “{insert disaster of choice}” is happening. You begin to look at flow runs and realize the flow doesn’t tell a particularly coherent story that is missing a lot of detail. You find yourself wishing the flow was written to be a bit more supportable. It turns out a lot of what your mind easily recalled 5 months ago is a bit fuzzy now and the day’s stress isn’t helping.
· Even worse, imagine you got the troubling email and didn’t write the original flow and you have to debug a similarly incoherent story written by someone else.
This is the motivation for the style guidelines
Top to bottom, left to right. In many of the world’s languages (of course not all), including English we look to the upper left corner to start reading and we expect to finish it in the bottom right corner. This metaphor is well-used in writing OO flows. I have seen OO flows that begin on the right, end on the Left or end in the middle. When you can, have your flows end on the right edge of the canvas and begin in the upper left corner. A personal choice - I like to put my desired outcomes at the top of the right edge and the less desirable outcomes at the bottom.
Keep your transition lines from overlapping. Transition lines that overlap are like a long program with a million “goto” statements. They lead to flows that are more difficult to read and debug later on. Flow transition lines that overlap are not a matter of prettiness and the more you “patch” your flow, the worse it becomes.
Put displayed flow variables in parens. Bread crumbs of the run of your flow are incredibly useful. Use flow variables, and when you do, put them in parens (). For example: Imagine you have a flow variable ${MyVariable} and you expect it to be a string like “OK” or “Error”. Imagine that you display it with parens.
MyVariable=(${MyVariable})
If ${MyVariable} is “OK” then in the debugger or flow run history, you would see:
MyVariable=(OK).
This might seem like a lot of extra work. Before you reach that conclusion, consider if your flow variable is not “OK” or “Error” but it is null (“”). If you don’t use parens, you see:
MyVariable=
If you use the parens you see
MyVariable=().
In this second case, I know the result is not a blank character and it is not a series of blank characters. It is null (or unprintable).
Suppose you are parsing strings from a list
${TicketList} = “IM420054, IM420057, IM420061, IM420065, IM420078”
If you strip values you might get ${OneTicket} = “IM420054” “IM420057 “ or “ IM420065” – depending on how you stripped, or how the data arrived. Consider these and decide for yourself what might be most helpful:
OneTicket=(IM420054)
OneTicket=(IM420057 )
OneTicket=( IM420065)
Parens help the “extra unhelpful little bits” stand out rapidly as an error. When you are having a bad day, and your stress level is a bit high, these little issues need to be really visible to you.
Use transition lines as the blessings they are. This includes helpful titles and meaningful descriptions. In most programming environments you have a log (file) that you use by adding log statements to your code. If you put in one log for every line of code and go into a loop, you can crash the environment by running the server out of disk space. On other hand if you put in one log entry for every 100 line of code and something fails, you might not be able to track down the source of the problem. OO has decided (for us) that in general, every operation in your flow is going to generate logging information. If you fail to use it, OO is going to experience the overhead anyhow and you just won’t get the benefit. As long as OO is going to make a DB call (to log the result) each time anyhow, you might as well take advantage of it. Tell a story that someone will understand when the flow runs. Make it clear. Write the flow, then as you are debugging it, ask yourself if the flow run result tells a meaningful story. The transition is recorded when your flow exits an operation, so tell what happened. If you did a string compare to see if a condition was true of not, show the conditions. For example:
Does (${PingResult}) contain (Ping failed) – No, the server is pingable, no errors.
If ${FlowList} contains a list of flows to be triggered like Check for Duplicates|Hourly Maintenance|DB Cleanup and you go into a list iterator, put the list on the transition to the iterator such as:
Preparing to iterate the flow list: (${FlowList})
In this case, the iterator is pulling ${OneFlowName} and ${OneFlowUUID} out of the string (using that wonderful OO strip capability). On the “has more” step (your main intended route) put the following on the transition label:
Launching flow. OneFlowName=(${OneFlowName}). UUID=(${OneFlowUUID})
In this case, the transition labels will tell the story of each flow you iterated.
Try to keep transition lines out of step names. This can be difficult, but it is helpful to be able to read a well-chosen step name. The example on the left is quicker to code, but the example on the right is easier to read and debug.
It really didn’t take too long to “shape” the transition lines (shift+click on the line and drag the “control point”) to move the lines out of the way of the step names so those well-chosen step names can be read.
Consider changing the titles of transition lines to better reflect what the flow is doing, as opposed to how it is doing it. Two examples come to mind:
In the often-used “Check null” the responses are “isNull” or “notNull”. While literally true, the Check null is how the flow works, not what it intends to do. In this case, the flow is checking a list to see if there are tickets. When the ticket list is null there are no tickets. It seem helpful to name the test in the form of a question “Are there tickets” and then to replace the (technically accurate but not as helpful) “isNull” with “No” and “notNull” with “Yes”. In this manner you know exactly what is being tested and you easily understand why a particular route was used. You don’t have to go through the mental processing step of “hmmmm, is null when we are done, or when we have more to do?”.
When you use the String Comparator (string compare) you may be tempted to use the default operator name “String Comparator” with the question you are asking and “success” “failure” with “Yes” and “No” or something else that reads well.
This technique is particularly helpful when Yes might mean “I found an error” or it might mean “the result is valid”. Unfortunately OO uses Green or Red for all these operations because culturally we think Green=Good and Red=Bad. Sometimes you have to test for a “something” to indicate an error and sometimes you test for a “not something” so “good” could be either red or green. Making the operation a question and the responses into (contextually helpful) answers you can make a flow more readable.
.
If you are going to loop, you can make it look like a loop. OO doesn’t do as much “looping” as a lot of programming languages, but when you do it can be helpful to provide visual cues. Here are a couple of ways to do loops.
Try to put transition labels near the “start” end of a transition line. When you get a lot of transition lines in a flow (or subflow) it is helpful to put the label near the front of the transition line so it points “away”. This is particularly helpful when you don’t have space to bend the line into an arc pair.
Consider numbering your errors. Those pesky little red “fail” blocks can overwhelm your flow. You can save space by using one fail block as an output from multiple operations. The key is to know how you got to one of them and to do this, label them with a shortcut like F01, F11, F10. In this case, F stands for failure and the number identifies the unique position.
Make a BIG DEAL out of UNANTICIPATED FAILURES. You can take red pathways that are expected. A string does not match so you take the red outcome as a matter of success. However, when you reach one of the fail points (as in the previous example) and it really is a failure it is helpful to make this fact very clear. In the description of the failure it can be helpful to put the transition label in UPPERCASE TEXT. Here are some examples of error messages I have never seen in Production.
UNANTICIPATED FAILURE F19 – UNABLE TO GET THE SYSTEM TIME.
UNANTICIPATED FAILURE F04 – UNABLE TO PLACE A VALUE INTO A FLOW VARIABLE.
These are errors you never expect to see, so when they happen you really ought to get attention for it. Unable to get system time is a problem that is very serious (like a JVM failure). WHEN YOU PUT YOUR MESSAGE IN UPPERCASE IT GETS ATTENTION.
It is difficult, but if you expect useful information to be in the failure blocks, it can be useful should the flow ever actually fail to see what happened. If a string is always going to have a value, put it in the failure transition step description. On “that difficult day”, you will be glad you did.
Sometimes, you will have a flow that only fails in Production and only fails under certain circumstances. When this happens, displaying trace or remnant data can be the best clue you have. This of the time you spend doing this as if you are maintaining the back box on your airplane. If there is a crash, this may the best data you have about what was going-on.