Management Tactics for the real world complex projects

A lot of management practices are used to optimize to current state of the projects and get the situation under control, but it does not always work as intended. Unfortunately, many people think that just applying the required process does the trick, however, in many cases it does not. I've published the presentation Management Tactics regarding that, and you can find the corresponding notes with the detailed explanations below.

Before going further please let me notice that management practices do work, however in many cases they're incorrectly used and that lead to frustrations and the articles like Why Scrum Should Basically Just Die In A Fire (it's a Web Archive copy, the original article has been removed). We live in the subjective reality (please see A guide to why your world is a hallucination), but the objective reality gives us the challenges which we may overlook due to differences between our perceptions. Let me try to point out few tactics how to deal with those challenges and succeed in your projects.

Points for Consideration

The main points are described below, and the rest of the presentation is just the examples of those points, including their reasons and possible solutions.

• There is No Silver Bullet

  Please keep your expectation low - we're dealing with the highly volatile factor (the human) and we can't plan/predict everything with the high accuracy. If some method says that it can deal with all the problems, please be skeptical. Good example - Zero-Bug Software Development. While it has some valid points, and can be applied for some projects, we have to take it with a grain of salt - bugs are still there, just called differently.

• Be Rational (e.g. legacy ≠ untouchable)

  While we prefer to think about ourselves that we're rational, it's not always true. Please try to answer a question - would you put all your fortune on a statement that any new combination of properly shuffled deck of 52 cards has never been seen in the history of universe? It's tough to say "yes", but actually 52! is a huge number, and the Solar System will disappear faster than shuffling will produce the same combination even if we're doing it with the speed of million combinations per second.

  One of the most infamous irrationality is "if it ain't broke, don't fix it". Environment is changing in many ways all the time, and the things which were working yesterday, could be broken tomorrow. Y2K bug costed billions of dollars, Unix Millennium Bug (2038 year) may cost even more. Using outdated technologies like Cobol creates a hiring crisis and money loss: Banks scramble to fix old systems as IT 'cowboys' ride into sunset.

• Do Risk Analysis

  However, being on a "cutting edge" may have a cost too. For example, nowadays front-end technologies move very fast, and it's very difficult to keep up the speed. Moreover, new changes may introduce new bugs, and some of the new technologies don't worth to be even used.

• Take Interest in Psychology

  We're working with people, not machines, and it gives us the challenges which we may overlook. Few examples are described below in other sections.

• Make Knowledge-based Decisions

  Intuition may fail us, and it's better to make decisions based on the knowledge. Surely, everybody knows the importance of prototyping and research, however let me notice another factor - transparency. We couldn't know anything, and sometimes it's useful to include teammates in the decision process because they may have insights you could be unaware of. Interdisciplinary studies gave us neural networks and deep learning technologies which affect the whole IT industry now.

Everybody Lies

While people prefer to think about themselves as honest, in reality it's not always true, and sometimes we lie and are being even unaware of it. Even criminals have self-justification and use a reasoning like thinking about themselves as noble and positive.

Reasons

• Personal/Financial Motives

  Please note that not only your staff may have personal (including financial) motives, but vendors and subcontractors too. Don't be a marionette of the marketing, and please don't think that if everyone (big players, your friends, fortune 50 companies) are using some technology, you have to use the same one. They might just be stuck with it or have irrational reasoning like "if it ain't broke, don't fix it". For example, Larry Ellison allegedly tried to have a professor fired for benchmarking Oracle, and while it doesn't mean that Oracle or MS SQL servers are slow, you should consider your own benchmarking before deciding which servers to use - it's quite possible that the published benchmarks are just the marketing.

• Laziness

  Laziness is not always bad, especially if a developer automates the processes, making it possible to be lazy later. We're not machines, and sometimes we just don't have a mood to work hard today, could have some family/personal issues thus couldn't work effectively. I think it's completely fine given the tasks are not stalled and finished in time. Moreover, it became pretty much common to reduce work hours to make staff more satisfied with job hence increasing productivity: Is It Time to Kill the 40-Hour Workweek? However, there will always be people who want to abuse the system, and just to be lazy. No process can absolutely help with it (without the price of everybody suffering), and it's a responsibility of managers to take care of it.

• Psychological Immune System

  It's not an official term, and I'm referring to TED Talks here: The surprising science of happiness. It's important to know that the role of unconscious could be much bigger than we expect, and we have to be ready to deal with it. One of the most infamous examples here - programmers' obsession with the technologies they use within the long periods of time. At some point, they can't listen to any rational reasons against the technology they like, because their unconscious protects them from the possible stress that could occur after realizing that they invested so much efforts into something flawed and not so suitable for their tasks.

• Dunning-Kruger effect

  Another infamous characteristic of many programmers - they're oblivious to any problems in their code, because don't have a competency to see them. Moreover, they're fiercely protecting their code therefore making any critique a personal matter to them.

Solutions

• Common Vocabulary

  Using clear terms may help with misunderstanding and reduces a risk that developers use the worst definition of the term thus allowing them to be lazy and blame their managers for being not clear enough. E.g., please avoid using generic "concurrency" term in the design documents, but use the concrete multiprocessing/multithreaded/clustering requirements.

  Using common code style may help with Dunning-Kruger effect - enforcing certain rules can make code much better, and while initially you may see some resistance, eventually (due to Psychological Immune System), developers will like it. Please note that strict enforcing works better (because it pushes Psychological Immune System to work) comparing with giving the options ("I'd like you to do something, but it's not required"), and even more, the latter may cause a prolonged conflict.

• Task → Artefacts

  The tasks should be always verified by the produced artefacts. If a task doesn't generate anything, then it will be abused. If you ask, "please do the research and make a decision regarding whether this functionality possible or not," there is a high probability that the researcher will spend weeks doing nothing and at the end says, "It's not possible." However, if you ask, "please do the research, provide me with the references you find and make a decision whether this functionality possible or not," the possibility of abusing could be much lower.

• History of Changes

  Tracking history of the task artefacts gives an ability to make the rational decisions about the performance of the developer. Being lazy for only a week could be acceptable, but being lazy for months could be a sign that we need to let this developer go. Having history of changes may help a lot with the decisions like that.

Shit Happens

A lot of management methods do not work well with the deadlines and other stress-related circumstances. Waterfall (and many agile) practices crumble upon it, and consequences may be catastrophic.

Reasons

• Incorrect ETA

  Having the accurate plans is almost impossible with the volatile factors like humans, however the business still requires some estimations, and surely it fails very often.

• Workforce Issues

  People may become ill, they have families with children, occasional vacations, and may even want to sabotage their work if they have some grudges against teammates or the company. All of those may lead to unintended delays and other problems with the projects.

• Uncontrollable Environment

  OS glitches, viruses, hardware failures, 3rd library broken updates (How one developer just broke Node, Babel and thousands of projects in 11 lines of JavaScript) may have catastrophic consequences to the projects you're working on and you have to be ready to deal with it.

Solutions

• Thresholds

  Having reasonable thresholds regarding certain metrics like latency, bandwidth, code coverage etc. may help with keeping situation under control in the events of stress-related circumstances. While having 100% code coverage is nice, during a deadline it would be very difficult to write the fully covered code. At the same time, not measuring any coverage could be harmful too, because some critical code may not be tested and you won't know about it. Having some threshold (like 75% coverage) could a reasonable compromise.

• Proper Tests

  All levels of testing are useful. You may have unit-tests passing, but a smoke test verifying that the program could be started could fail. Integration tests may take a while and during deadline you may have problems running it each time the code changes, and having minimal smoke tests may help in this situation. Continuous integration with the isolated environment (with source codes not being up-to-dated, but cloned each time from a repository) could greatly help too (e.g., for verification that all source codes are actually committed into the source tree).

• Collaboration

  Please use the suitable tools for collaboration. For example, having everything in tickets makes transferring the knowledge no mean feat, and forbidding to use IM could make the communications flow very slow. On the other hand, if it's allowed, it's tempting to discuss all work-related matters using IM, and it's a responsibility of the manager to track it, and help people to understand the proper usage of tools at their disposal.

• Rules as Intended > Rules as Written

  A lot of managers put the Agile processes before the people, enforcing them to follow the procedures and waste their time for formalities, not the actual work. However, the creators of Agile methods didn't want it, and it's expressed in their Agile Manifesto: Individuals and interactions over processes and tools. Sometimes we need to bend the rules, because they do not fit the current situation and may cause more harm than good.

Thresholds

It's almost impossible to create the perfect software, and even if it's close to perfect in one environment (with predefined OS/hardware/usage scenarios), it may fail in another environment. Right environment and tools can help us getting better results (hence, use higher thresholds), and it should always be considered before starting the project. For example, when switching to slower but feature-rich technology, you can do A/B Testing regarding what latency is acceptable for users. You don't always need to use the best technologies (it has some cost in the most cases), and using a little bit inferior but still acceptable technology could make the development much cheaper.

History of Changes

Retrospective analysis of the project (knowing not only the current state, but the history of events concluded in this state) can increase the prediction accuracy and help with planning. For example, when you're researching the features similar to the ones you have worked with before, you can use the retrospective analysis with the help of VCS (i.e., how many days you've spent on similar features based on number of days having commits involving those features; and deviations can help with calculation of pessimistic/optimistic time boundaries).

Almost all types of activities produce the artefacts (directly or indirectly - for example, we can just register the activities being done with audio/video capture). Brainstorming (i.e., research) tasks can follow the scheme: the whiteboard hypotheses -> photoshoots of the whiteboard (+ put in VCS) -> prototyping using programming or SQL (+ put code in VCS) -> results (+ exported as text and saved in VCS) -> back to the whiteboard or rollback to the previous hypothesis/prototypes/results.

Managerial Tasks

While it's widely believed that the task trackers can be used directly for planning and reporting, it's not an easy task to keep the data there consistent and useful. It's much better to consider the trackers as tools for convenience, and don't completely rely on them for reporting.

• Don't trust the tracker, but only artefacts

  If managers blindly believe the tracker, it leads to abusing it (for example, if performance is measured by the number of closed tickets, the developers and testers begin to open dozens of tickets even for minor issues).

• Done for developer ≠ Done for manager

  All people are different, and managers and developers have different understandings of the requirements. A developer may think that the feature is implemented, but the manager - not (usually managers have better understanding of the business requirements and can notice the use-cases which were overlooked by the developer). Also, if managers verify the features implementation, they can see the progress closely thus the probability of getting the situation out of control is getting lower.

• Checklists could be artefacts by themselves

  Developers and managers have their own checklists: developer's is based on real artefacts (code and documentation), manager's is based on the manager activities (like verifying that the business requirements are taken care of). It allows us to find bottlenecks by analyzing the dynamics of the changes, also time-based checklists can be re-verified later (for example, if CI has stopped delivering correct results and the manager has been changing the checklist based on those results, it has the high probability that the manager has been just lazy and put the ticks without any verification). It’s not needed so often, but can improve the discipline in the company (when managers know that their work can be analyzed later).

• Reports are based on artefacts and their dynamics

  Task trackers do provide valuable information, but aggregated reports from other sources like VCS commits could provide more accurate and useful results. Please note that you may use task trackers indirectly (via reading its meta information and databases), hence you may get better insights (see the examples in the presentation).

Scalateχ \src: MgmtTacticsScript.scalatex