I’m covering three design principles we use at Kitepipe. These are:
- Empty Pipe
- No Black Hole
- Part of the Business Process
This post will consider No Black Hole.
What Does "No Black Hole" Mean For Data Integration Design?
No Black Hole refers to the widespread practice of thinking about integration as “data synchronization”, and making it invisible to the user. Making it a Black Hole.
Much of what we do at Kitepipe is transaction integration, not data synchronization. In transaction integration, the payload has high value - and somebody, maybe many somebodies, care if the integration completes successfully. These might be Sales orders, shipments, or employee status updates. So we work hard to make the results visible to the user. In all cases we notify about errors, and often about successful integrations, as well.
Kitepipe's Techniques for Integration Status Communications
We use a range of techniques to communicate to users about the status of integrations. Techniques include:
- emails that alert to errors
- emails that alert to success
- logging to a log database or application
- posting status back to the source application
Our experience is that users are very interested in integration status, and welcome “rich information” about the status of the transactions. Giving users prompt, easy to access status information also has another important benefit - it takes the IT or Data or Integration team out of the data correction loop. Users can see data problems, fix them, and rerun them.
But only if the integration is part of the overall business process - which is the next principle.
Read Our Other Articles on Data Integration Design
Three Principles of Data Integration- Empty Pipe
