Managing your tools

After a few decades of ongoing global software development, there is a tool for just about everything.  And for everything else, you can develop one yourself (if you know how).  And since you want everybody to share the stuff they do, all those tools have some sort of central server.  Be it a database server, web server, file server, everything has a server nowadays.  The downside is, after a while you have lots of applications running on various machines (usually virtual machines) and it’s getting harder and harder to keep an overview.  Every day somewhere a disk is filling up, or an application crashes, or something else goes wrong.

At DN we have dozens of servers running.  There’s our our web site, TeamCity (for CI/CD), BitBucket, monitoring tools, etc.  After a while I was losing one or more hours every day to try to keep it all up ‘n’ running.
So how to fix it?  We solved it by containerizing our applications and running them using a central container orchestrator called Marathon, combined with a great monitoring tool called Prometheus.  It works great, and now I have a shiny clean dashboard where I can see all our running applications in a single place.
In a future post, I will elaborate on the details of our architecture.

Author Wout Neirynck is owner of Debreuck Neirynck and an experienced agile developer with strong focus on TDD 

Don’t Institutionalize your problems

As a small software company, we’re constantly evolving.  Especially in a sector like ours, where change is very often driven by external factors.  But that doesn’t mean that we don’t have to cope with problems that other (more “traditional”) companies also face.

A few years ago, we were going through a turbulent period.  This was some years after we did a lot of new hires, and it was beginning to show.  We didn’t have the time to educate the people we hired to work on the level we expected, and as a result, the quality of the code was not as it should have been.  This in turn resulted in lots of bugs that needed to be fixed and we couldn’t bill the customer for them. 

We (the management) were of course not happy with the course of events, so we wanted to stay on top of it and avoid further escalation.  More specifically, we tried to keep the amount of time spent fixing those bugs as minimal as possible.  We have developed our own tool for internal use to track effort time spent on the various issues, and in those days we divided those issues into stories (aka “new stuff”) and tickets (aka “bugs”).But effort spent on tickets and stories kept escalating, and that meant that the deadlines were being missed. 

Our already grumpy customers were becoming more and more agitated, so a solution had to be found.  One of the proposals was that next to the estimates that development gave for each issue, we should also add another metric, the effort that was still expected to be spent on the issue.  So let’s say that there was a story that a validation check on the e-mail field of the input screen for a new user needed to be added and that development estimated they needed 16 hours (or “points”) to implement it.  After a few days, maybe it still wasn’t finished (for whatever reason), but now the original estimate was exceeded and so the extra metric kicked in and development would need to do an additional estimate to indicate how much time there was still needed to actually finish the story.  And they would have to update it daily, until the story was done.

At first it seemed like a good idea, it would give us the information we needed to tell the customer whether we would be able to ship in time or how much overtime there would be (or which features we would need to cancel or postpone).  But very soon it appeared to me that we weren’t actually solving anything.  We were just adding a new layer of bureaucracy to the “failing” system.  The original estimate was wrong, and overtime was being spent.  We didn’t need any additional (possibly wrong) estimates to tell us how wrong we were.  We just needed to fix the original problem: improve our estimates and improve the code quality to avoid bugs!

Another example (maybe more familiar to some) could be that management requires a late team to issue a daily “status report” until the situation improves.  They’re not really adressing the problem, instead they are just giving the overburdened team just more work and more stress!

So once that dawned on us, we canceled that feature and ever since then we call something that just registers the problem instead of fixing it “institutionalizing your problems”. In another article I’ll elaborate on how we managed to fix the original problems we had.

Author Wout Neirynck is owner of Debreuck Neirynck and an experienced agile developer with strong focus on TDD 

Why we don’t talk about the behavior-stuff we can’t handle?

or: Behavioral modeling within a bounded business context.

We humans are amazing. Truly a wonderfull mechanism of bio-enginuïty to deliver a living body that, with some basic nurtering, lives about 75 year and is able to provide all kinds of output: (not limited list:) labor, intelligence, creativity, love, hate, anger.

An important dependency for evolution is to ‘formalize’ some experienced knowledge. Most obvious example is ‘language’: the existing of language makes it possible to evolve much faster in a human life where language exists than in a ancient life where it doesn’t. The existence of the internet changed the access to knowledge in general and therefore shortens the time needed to consume that knowledge.

How wonderfull we find ourself and our evolution, it stroke me recently how strongly we evolved in some area’s (science, mathematics, bio-engineering, …) and how “un-evolved” we are at other area’s (again: not limited list) : emotions, politics, worldpeace, long-life marriage…

Some major properties we can assign to the ‘hard’-science world (and not present in the ‘soft’ area):

  • unbiased data history
  • symantic clarity
  • general excepted formal models
  • exempted from reality

The last property ‘exempted from reality’ seems obvious. We do not define mathemathical models in a holistic approach. We create a seperate world with rules and language not interrupted with the chaos-input of the world. (dixit Nick Cave : ‘Outside my window, the world is gone to war’)

Perhaps that’s why we design Business Processes as if their are extracted from the real daily business: let’s take an example:

Simple 2 step approval process.
It seems obvious because we can’t model the real behavior of this approval step, we don’t know who is, what are, how fast, …. And even if we tried, chaos/reality is changing every split second.
However, why is this? Don’t we have those properties that seem to be missing?

  • unbiased data history: we are overwhelmed with data, certainly we can detect the approval point in our ERP system
  • symantic clarity: unknown input can be expressed in clear averages, and be replaced in production by real time event streams.
  • formal models: see below : SD is based on pure functional math and has been around since 1950’s.

The traditional BPMN model is not answering real Business Questions like :

  • how many approval requests can my business handle?
  • is there suffient staff for approval step 1, and if not, what is required for my business?
  • what happens to my error rate if we would kill the second approval step?

We don’t talk about these questions or we try to avoid them. We leave those answers to Operational Management that simple needs to see what happens and react to it (=the behavior). That’s what they’re paid for…

The answer is to be found in the world of Systems Thinking, espacially System Dynamics, this model expresses the behavior of the process:


Run this model with a given set of values of the ‘outside’ world, the answer is defined:

Conclusion: the mystery of long life marriage is not solved, but we can model the behavior of business processes, and replace process-implementation-trial-and-error approach so we can talk about the business-stuff we can’t handle !

Spread the word.
Wim.

Exporting your problems

Have you ever noticed how we try to solve one problem, but end up creating new problems?  Because of the industrialization and accompanying population growth in the west, we chopped down most of our trees.  This in turn created a shortage of lumber.  So the government in lots of countries installed a ban on chopping down more trees.  But they did not provide any alternatives to lumber.  People needed (and still need) wood of all sorts and sizes to do stuff.  Provide heating (long ago), build houses (a little less long ago), build furniture (we still do that), etc…  So what happened?  Since chopping down trees was now illegal, we started buying the trees from abroad.  And for every tree that we imported, a tree in another country was chopped down.  Which means the deforestation got exported.  So that’s how you end up “exporting” your problems.

We here in the west have gotten pretty good at it.  Most of the deforestation in the 3rd world countries is driven by demand for wood or other products in rich countries.  We want to eat more beef, so we need land to grow the cows that provide the meat.  Over here, land is scarce, but in Brazil it’s not.  Just chop down a few acres of rainforest and tada!   You have more land to grow cows.  Unfortunately, the land is not very fertile and after a few years you need to chop (or burn) down more forest.Or recycling.  The government forces companies to recycle electronics.  But that’s of course way too expensive over here.  So the solution is easy: put all the broken and obsolete electronics devices in containers and move them to poorer countries.  There the people will recycle them for a lot less money.  And since the environmental and social legislation is a lot less strict over there, we can save even more money on social welfare and safety equipment!  Again, exporting our problems.


But we’re also doing it on a much smaller scale.  Lots of companies want to attain a “greener” image.  And one of the ways to do that is to reduce power usage.  Now everybody has some sort of private server park where all their applications run and their data is stored.  And that uses lots of power, so it would be great if that could be reduced.  Or even eliminated!  That’s where cloud providers come in: they allow you to migrate your applications (or entire servers) to their data-centers.  The result is that you don’t have to pay a big electricity bill and in turn pay some service fee to Amazon, Google or any of the other providers.  Great, isn’t it?  Yes, but you still haven’t reduced your actual power usage!  A server in the cloud still uses power.  Yes, maybe there is a slight reduction because of size optimizations, and they use more modern servers that use a little less power.  But we don’t really know how big a difference it makes.
It would be great if we would have a better way of measuring things like that. 

One positive effect of moving to the cloud is that cloud providers benefit from optimizing their power consumption, so even though we still export our problem, the global result may be a little less negative.

Author Wout Neirynck is owner of Debreuck Neirynck and an experienced agile developer with strong focus on TDD  

Survival of the fittest (solution)

or “Do we really need event driven solutions ?” 

Last week i received a question if the use of an event driven architecture was desirable in a specific project. The architect was in fact looking for arguments not to use it, because in his opinion it was rather a ‘simple’ solution, so why create the effort to push all request/replys through message-bus, monitor it, make it failsafe, load balance it and so on. 

It could only make the solution slower and complexer. My answer was : “there’s no argument possible NOT to use events in the solution if your solution is a Entreprise Business Solution” In other words, if it’s ‘core’ you need to use it, is it ‘throw away/gartner mode 2 development’ you don’t. 

To my humble opinion, software creation is all about getting as close to real life as possible. If you do it right you will create a durable, flexible and robust system, just like human life on earth is. Humans (and every life form) adapt to context, respond to their habitat, we go on, change, learn from our environment… 

So what’s real life? If you would model real life, which entity would you think of first? TIME Without notion of time there no real life as we now it. We need it to express everything else. To express anything you need to put it on a timeline = AN EVENT So even before thinkin’ about relations, processes, states, etc, there all are useless without the notion of the Event in Time. Any core software solution that needs to ‘live’ within a Business Landscape with other applications, systems, digital entities,… needs an clear event-architecture to survive. As we now are able to process data as it occurs (compared to post-persistence processing) by means of handling the datastream, we’re actually processing events in (current) time. 

Halleluja, isn’t that getting pretty close to real life ?, actually ‘living’ the day by processing events as they occur ? So basically everything that ‘lives’ has a constant event-stream, if not, it’s dead. So please build ‘live’ applications, not dead ones, so they can stand straight and survive in the ever-changing business-context ! 

Even you’re mother in law is (nothing but) a (non-linear) eventstream, but don’t tell her that the next time you see her 😉 

Author Wim Debreuck is owner of Debreuck Neirynck and an experienced Business Analyst with strong focus on System Thinking

Author Wim Debreuck is owner of Debreuck Neirynck and an experienced Business Analyst with strong focus on System Thinking

SOFTWARE-INGENIEURS, OF TOCH NIET?

Ik las laatst een opiniestuk over software ingenieurs.  Of althans, hoe we onszelf graag noemen.  De auteur had daar enige bedenkingen bij, en volgens hem mogen we die term niet op onszelf toepassen.  We maken er immers te vaak een boeltje van, zo parafraseer ik zijn argumentatie.  En daar kan ik hem helaas geen ongelijk in geven.  Al te vaak komen mislukte IT projecten in het nieuws, tientallen miljoenen euro’s gaan zo in rook op.

Maar toch vind ik het te kort door te bocht om de IT sector direct al af te schilderen als een bende cowboys die nauwelijks weten wat ze doen en maar met het geld van de klant brassen zonder gegarandeerd resultaat. ….  

 DOWNLOAD HIER HET VOLLEDIGE ARTIKEL

Wout Neirynck is zaakvoerder van Debreuck & Neirynck en heeft een jarenlange ervaring in Test Driven Development en software bouwen op een Agile manier.

How to expect the unexpected?

Fact : we are getting more and more impatient

  • if we buy we can’t stand delivery times that exceed one business day (soon we will demand delivery within 4 hours)
  • if we travel we pay extra to get there faster (highway taxes, costly airplanes tickets for direct flights,…)
  • if we consume news we want short articles (tweets – facebook – blogs – …)
  • if we rather chat then send mail, because you get reply sooner (succes op channel oriented chat apps, whatapp, slack, …)

I could go on, but fact is that we can’t stand waiting (and it’s getting worse). Unfortunately some things take time and the instant-satisfaction-high is not always possible. So how do we manage this impatience, this waiting for .. ?

The answer

Countdown traffic lights are heaven, they give you peace of mind, and extra time to check you cellphone, close you eyes, whatever. Every country should have them. They let you expect the unexpected moment when the lights turn green. Don’t know who came up with the idea but he/she is a genius!

In an ever faster moving business world and with more and more impatient customers, companies need to provide countdown traffic lights for those moments that the customer needs to wait. The customer only wants to wait when he/she knows how long he needs to wait. Companies that fail to manage this, will be bypassed by those who do.

Promising delivery next business day and ringing the doorbell at 6 pm is experienced as bad service because the customer was hoping to get delivered round midday (as the previous delivery was at nicely dropped at 11 am)

Business analysts need to investigate how they can provide an answer to this. How can we inform the customer about the progressing state of his/her request? How can we detect the ‘perception’ of bad service even if we meet our SLA? How can we predict in an non-linear environment?

System Dynamics to the rescue !  For fun, I designed a model to predict how long it takes to get out the department store at certain time. It’s still beta, but even now you see the non-linear behavior all around.

Imagine a reliable countdown display at each cashier, then you’ll never would pick the wrong one !

Don’t tell me it doesn’t happen to you 🙂

Author Wim Debreuck is owner of Debreuck Neirynck and an experienced Business Analyst with strong focus on System Thinking.#system dynamics#business analysis#data driven#system thinking

Old school meets New

Data Driven Business Analysis

Fascinating how the value of knowledge is dependent on the context of time it exists in.

When i learned the world of ‘Systems Thinking’ and ‘Systems Dynamics’ a couple of years ago on a quiet vacation in Italy, I was amazed of the ingenuity of the approach and the fact that it was available for all human kind as of the year 1950 or so. Why did we not use it in our every day (business analyst) life?

It took me a while to figure it out, but the simple answer is: the availability of data. A System Dynamics model needs detailed grained data to perform well, or else it’s just a theory.  I tried to ‘materialize’ the work of Russell Lincoln Ackoff and Jamshid Gharajedaghi (check out their literature), and isolated the approached within a specific domain:  Container-transport for Bridgestone in the port of Zeebrugge.

What if we coded the model into our plannings-engine?  The model predicts bottlenecks, if we can use the output of the model as an input parameter of the application, we could automatically decrease transport activity if we see a unloading capacity problem in future (for example).  We did, and this blog is too limited to describe the possibilities of what we can achieve with this approach.

The Old School knowledge comes to new life and generates an enormous business-value in today’s Big Data all-around-us world!

Last week i presented the case in the BA & Beyond Conference 2018 in Antwerp ( My presentation can be found here https://prezi.com/view/y3Rq0f54pQc9RTdI5Hct/ ) , and i was glad to see that other people also discovered the value of systems thinking. Click here to check out the blog of Adrian Reed

I hope that people get inspired and more and more evangelists promote the concept. If we want to design smarter systems we need to have a smarter approach to design them!  Let’s use all the knowledge human kind produced and look at it in today’s world.

Just like batteries boosts cycling all around the world, Big Data makes System Dynamics the smart-engine in every project !

Bezoek van EU-parlementslid Tom Vandenkendelaere en schepen Nathalie Muylle

We hebben het genoegen gehad om EU-parlementslid Tom Vandenkendelaere  en schepen Nathalie Muylle van Roeselare te mogen ontvangen.  Na onze presentatie van de gerealiseerde projecten digitalisering in de voeding- en landbouwsector, vatte Tom het samen als  “Met technologie naar een bewuste en duurzame landbouw” .  

We waren ook blij te horen dat er in Stad Roeselare grote interesse is voor innovatie.  Stad Roeselare steunt ondernemen, ook in de digitale wereld van morgen!

Bedankt voor de interesse!

DN Talk – BLOCKCHAIN

Op woensdag 16/05 mochten we terug een tiental ondernemers begroeten voor de tweede DN Talk.  Deze keer was het de technologie Blockchain die we hebben toegelicht, dit op een begrijpbare taal vanuit het oogpunt van een ondernemer.  

Mario Demets gaf ons zijn mening over hoe deze technologie onze manier van zaken doen zal wijzigen.  We onthouden de opkomst van de netwerk economie, waar  uitwisseling van gegevens op een veilige en betrouwbare manier zal kunnen gebeuren.  Blockchain zal zorgen voor een vertrouwen tussen zowel bedrijven die samenwerken als tussen bedrijven en de consument.  Mario besloot zijn talk met het feit dat blockchain zal bijdragen aan een nieuwe manier van business doen en niet meer zal verdwijnen.

Tijdens de talk werd blockchain uitgelegd aan de hand van een door Howest geproduceerd filmpje –  dit kan u hier terugvinden op Youtube.

Maak je eigen website aan bij WordPress.com
Aan de slag