When people think about DevOps they often think about CI/CD pipelines and version control, but when your integration fails overnight the focus isn’t going to be on how clean your deployment pipeline is. Instead, the focus is going to be on how quickly you diagnose the issue and how fast you fix it.
DevOps brings together software development and IT operations into a single, collaborative approach, rather than treating deployment pipelines as the main focusIt encompasses the ideology that maintaining and running integrations is just as important as building them.
When it comes to Boomi as a Platform, many of the core features of a CI/CD pipeline are already built in, so there is little that Boomi developers must do to maintain a ‘clean’ deployment pipeline. Making a Boomi process supportable on the other hand, comes largely down to the way you design and build it. In this blog post we’re going to look at DevOps in Boomi, and how you can design and build your Boomi processes with supportability in mind.
Make Use of Environment Extensions and Avoid Unnecessary Hardcoding
Time and again we see developers that hardcode all their values inside their Boomi processes. While hardcoding has its place, it can often introduce challenges where even a small change in Production triggers a full development cycle. This involves a developer having to first update the Boomi process, package and redeploy it, test it and then navigate change management to get it redeployed to Production.
Instead, any values that are likely to change should be configured as part of a Boomi Cross Reference table or extendible process property during development. By correctly externalising these values, support teams can update them directly through Boomi Environment Extensions in Production, avoiding many of the traditional steps that typically accompany change management. This approach dramatically reduces effort and speeds up issue resolution time by allowing for the separation of operational concerns from implementation logic.
Figure 1 Updating a Cross Reference Table in Boomi Environment Extensions
Use Document Tracking
Boomi’s Process Reporting is a useful tool, but its usefulness depends on how well you configure your processes to leverage its built-in capabilities. One way in which Process Reporting capabilities can be greatly improved is through adding Document Tracking to your Boomi Connector Operations. By tracking meaningful identifiers (e.g. order numbers) you can effectively leverage Process Reporting to track down specific process executions for analysis. This is particularly helpful in cases where you are given a particular identifier and asked to see ‘what went wrong’. It also allows you to verify if Boomi ever actually received and processed the data in the first place.
It’s important to note however, that not all Document Tracking is useful. There are best practices when it comes to Document Tracking that should be followed to get the most out of it. It’s certainly not fun getting a timeout or connection error in your Boomi process and then seeing the dreaded ‘failed tracking documents’ error popping up in Process Reporting. This not only looks ugly but also obscures the true error from being visible to the Operations team.
Figure 2 Configuring a tracked field using a Dynamic Document Property
Log With Purpose
Adding logging to your processes is great but make sure to log with purpose and only log the data that really matters. If you log too much data, it can become confusing to look at and obscure the Operations team from seeing the data that matters. Alternatively, not logging enough data can be a hinderance to the Operations team and stop them from being able to see the true cause of an error.
Effective logging is about context and should enable you to find out what data was being processed when the failure occurred and where the failure occurred within the process flow. Common examples of data that is useful to log includes logging the Last Successful Run Date value the process used when executing, and any unique identifiers that are key to the process. In some circumstances it is also useful to log request payloads but be careful in doing this that you’re not logging confidential information. You should also be wary that Boomi will truncate log data that is too long.
Figure 3 Logging the Boomi Last Successful Run Date
Make Your Processes Readable
When designing and developing your processes you should make them easily readable. This reduces the time it takes the Operations team to work through issues when troubleshooting your process. Assume when developing that you are not going to be the person supporting the process, and label and name your process components according to a ‘Salesforce to ADMS Create Outage’ to indicate the integration’s source, target, action and entity. Make use of Boomi’s Notes feature and add notes to your process components and your process itself, and reuse common subprocesses and process components where possible.
Figure 4 Adding notes to a Boomi Connector Step
Design Processes with Failure in Mind
For every process, it’s not a matter of if it will fail, but when. All processes will experience failure at some point, but it’s how that process recovers from failures that makes or breaks it.
As such, you should always aim to design your processes with failure in mind by thinking about what the likely points of failure are, considering how can things be rerun, and understanding what the impact will be if some data is lost or duplicated.
A process that is designed well should allow documents to be rerun as required, minimising the amount of data that is processed as duplicates. Quite often this means giving the process the ability to rerun individual documents one at a time, instead of forcing the process to rerun the entire batch. One of the easiest ways to allow processes to rerun only errored documents is to make use of the Boomi Start Step to rerun errored documents using Boomi Process Reporting Operations teams can use this feature without fear of duplicate processing and with no specialist knowledge of the integration process required.
For some integrations, duplicate processing may not be a problem, but you still need an easy and reliable way to rerun the process when there is a failure. Depending on the process, you may be able to do this by simply utilising the built-in Boomi Last Successful Run Date. This allows you to automatically rerun processes from point of failure without having to even change anything.
Figure 5 Rerunning a document from the Start step
Use Effective Error Handling
Make sure to use the Boomi Try/Catch step purposefully, and don’t overcrowd your process with nested Try/Catches. Nested Try/Catches can make the error appear to ‘bounce around’ the Boomi process withlogs and make the actual issue harder to find. Additionally, the should be used to gracefully handle errors that the Operations team don’t need to see, so that they don’t waste time looking into errors that don’t require any action to resolve. It should also aim to retry transient failures so that the failed parts of a process can be retried automatically.
Effective error handling also means being consistent with your error handling approaches, so that the Operations team knows what to expect. Consistency could mean reusing a or making use of Boomi Platform Events. Whatever approach you choose, any error notifications sent out should be useful, and include a description of the error, the time of the failure, the process name and any relevant identifiers for failed documents.
Finally, make sure to return HTTP error responses for operations with common points of failure, and build in the right logic inside of Boomi to handle those error scenarios. It’s very frustrating getting an error in Boomi that is not gracefully handled, and all you can see in the Boomi logs is ‘500 Internal Sever Error’ or ‘400 Bad Request Error’ with no real error description of what went wrong.
Conclusion
DevOps is often thought about in terms of how efficient you can make your CI/CD pipelines, but in the Boomi platform where most of the features of CI/CD are already built-in, the focus on DevOps is better spent on how you can make your Boomi processes more supportable when they fail. After all, no one’s going to be checking the cleanliness of your deployment pipelines when your processes are erroring in Production.
There are many ways to make your Boomi processes more supportable, including designing them with failure in mind, making them easily readable, using effective error handling and logging and making use of document tracking and environment extensions. All these things help Operations teams to find and diagnose errors easier and quicker, leading to better recovery times and outcomes in Production.
To find out more about best practices for building your Boomi processes, check out Adaptiv’s free e-book here or get in touch with Adaptiv today.
