Going Agile      

By RAR                

It seems to me that when we experience big changes in our broad, shared experience with the world, it is usually the result of tribal tinkering. Somebody will come up with an idea and develop it until others see the value in it, and they adopt whatever behaviors are associated with the big idea, and because people have the following characteristics of sheep, you get adherence to the popular behavior until it becomes a group norm. Outside of behaviors that are reactions to natural disasters, there is very little else that shapes our experience with life other than the reverberations we feel from the machinations of our fellow man.

In the tribal world of software development, the biggest reverberator in the lives of most of its groups is “agile development”. It is the most logically conceived idea that you will ever encounter, and the most difficult for anybody to explain. That grey foggy area of understanding is like a John Carpenter phenomenon, populated by autonomous knifing pirates who are having devastating impacts on certain segments of the workplace jungle.

The idea of agile development came into existence in the 1980s within the DuPont Corporation, but it was formalized in the software industry in 2001 when a group of industry figures gathered at a resort in Utah to draft the Manifesto for Agile Software Development. That created some pillars on which the agile process would stand:

•  Individuals and interactions over Processes and tools
•  Working software over Comprehensive documentation
•  Customer collaboration over Contract negotiation
•  Responding to change over Following a plan

In practice, agile development is a process in which the aspects of a development project are divided among semi-autonomous development teams. Every piece of software consists of a range of functions, and developing those individual functions is assigned to individual teams. Led by a Scrum Master, the teams set short-term development objectives, with 3-week long sprints focused on developing their assigned function to the next milestone point. Teams meet daily (separately) to discuss progress and share ideas, and at the end of the sprint period they may release a new version of the still-in-development function into the still-developing software product.

While Team A is working on their part of the software, Teams B through Z are going through the same cycle of sprints to develop those parts of the application for which they are responsible.
As there is a significant interdependence among these teams, a bridge is needed to manage cross-functional communications.

In an Agile environment, cross-functional communications is largely done through program managers and ascending layers of middle management, but for the development team members, communications are largely managed through the use of tools like Rally.

Rally is a project management software that users update constantly to report on the status of various aspects of application development. It gathers together “User Stories” that define situations encountered in testing the application as written. The User Stories define how the software product should be modified or further developed to achieve the desired user experience. Rally provides schedule information for each aspect of application development, along with status reports designed to communicate how well things are going for each aspect of the project.

Communication is always the problem link in any organization, and what you will often hear from workers at the agile team level is that they don’t necessarily know what the other teams are doing.

For one thing, the Rally reports – which are daily snapshots of project status - often contain inaccuracies.
Looking back up at those four pillars undergirding the Agile Manifesto, one may recognize a subtext underlying the whole concept, most obvious of which is that agile development creates a competitive environment. It pits teams against one another, not in terms of what they are producing but rather in the way in which they are doing it. Teams that are struggling to meet their sprint marks are not likely to advertise this to the broader team. It is easier to apologize later, when you move your User Stories to the next sprint, to be resolved in that next three-week period. And unresolved User Stories may live on in this way from one quarter to the next.

Sometimes teams see no benefit in falling off the development schedule to wait for some lagging unit, and so they release their work even while teams with inter-dependencies are unable to release as scheduled. In big organizations, where there are a lot of agile development teams, you tend to get a lot of privately-held management tools designed to provide a real-world assessment of project status. You get people capturing the reality of where they are at in the development process through informal documentation, often in Excel or PowerPoint. Developers pull information out of Rally to create tools that better represent the truth of what they are dealing with, but mostly for their own use.

Software companies were early adapters of agile development, but with the advent of cloud technology the number of companies engaged in software development has skyrocketed. You get companies who are not known as software developers doing software development, and to support the needs of those latecomers there is an industry in agile consulting, which is a big feature offered by the Big Four consulting firms. Firms like Deloitte will bring in a team of agile consultants to help the company’s nascent development teams get organized into agile groups, and then they will mentor the whole team through the agile process.

Agile development has largely replaced the earlier “Waterfall” style of development, which put product creation under one team that shared resources and didn’t release any aspect of the product until the entire product was completely developed and validated. Waterfall developments characterize the type of design and development that very nearly wrecked the American automobile industry, where developers would take years to develop a new product only to have it hit the streets when the time for its design had already passed. (The rate of change in human perception and in human attachment to particular styles and fashions has expanded exponentially with the widespread use of the Internet.) These are hugely expensive mistakes, and so big manufacturing concerns started to look at ways to exploit a more agile approach to product development.

The Resulting Agility

Among the most profound changes that we have experienced in modern life is a focus on reactive behavior that became really profoundly impacting with the widespread use of the World Wide Web. Product developers suddenly had access to all kinds of data about the ways that consumers used and viewed their products, and so they sought ways to make it possible for them to quickly respond to what they perceived as customer demands.

That was huge. In an earlier time, products were developed by visionary designers who created products to meet unmet needs in the marketplace. U.S. manufacturing went through an early golden age of development when product designs were focused on style and durability; beautiful things were produced to last. Pushing new concepts into the market place was not really the first order of business. American culture was not geared to reactive change; to the contrary, it was geared toward classic designs that survived short-term fads and retained standards against which other products were judged.

Most people over the age of 50 can almost mark the year in which they saw the world change. The automobile industry was the canary in the coal mine. Around 1970, car makers began producing cars that were different from previous generations. Distinctive styling all but disappeared, outside of what was offered in some luxury models, and Americans got used to driving around in boxes that were designed to be cheap to buy and fuel efficient.

What was really happening was an early agile development process that swept through the auto industry and transformed it into a manufacturer of jigsaw puzzles. Cars could be compiled from an array of compatible parts, the tradeoff being that they had to all fit within a small group of body styles. Some things, like the safety features of cars, benefitted from this approach, but overall car design became boring and it has never really come out of it. The industry continues to be responsive to customer needs, and as fewer and fewer customers can afford anything more than low-end automobiles, the world has been flooded with low profile designs.

Most people over 30 can tell you that this same phenomenon, skewing toward unimaginative design, has also taken over the information technology sector. Like the automobile industry, the information technology sector is really dominated by a handful of big players who set the standards for, and often even control access to, the Internet. What Web designers can do in developing websites is more and more restricted by technical options, and by dictates in their realm analogous to the software industry’s Agile Manifesto. In its short history, the World Wide Web has changed character before our eyes. Gone are the garish, awful website designs that characterized the early years of the Web, replaced by cookie-cutter clean designs focused on optimizing the site’s visibility to search engines. And have you noticed that there is really only one of those: Google? You can use Bing and Yahoo and others, but there is no wide-open market in search engine development, as that, too, has been culled to only a handful of survivors, only one of which really matters.

Agile development supposedly puts emphasis on individuals and interactions, but that can mean a lot of things that have nothing to do with personal relationships.

In an earlier world, businesses placed a value on customer service, which often amounted to nothing more than provision of basic assistance delivered by someone behaving in a pleasant and attentive way. With the advent of the Internet and all of the data resources that came with it, customer service has been replaced with access to resources, and not necessarily resources provided by the product manufacturer or service provider. The do it yourself era had begun. People who had gotten used to pumping their own gas, became further use to ATM machines and online banking and purchasing. The world became flooded with electronic devices, and less personal in the process. More and more the support for those devices fell to users sharing their experiences in product user forums. Product documentation, per the third pillar in the Agile Manifesto, became a low priority in the new world where you could simply be encouraged to ask an online friend for help. The expectation was that your software products were expected to work, obviating the need for a lot of expensive-to-produce documentation and customer support.

The good news is that agile development has produced a lot of software that does work, because if it doesn’t there is very quickly a new release fixing the problem.

Making agile work in the workplace is something else again.

The Developing World

Sort of hidden among those pillars of the Agile Manifesto is something that is very precious to those honchos who got together at the Snowbird Resort to dream the whole thing up: intellectual property rights.
In an agile environment, very few individuals actually see the whole of the product that they are building, and even the visibility that they do have into parts of it is likely not deep. That means nobody leaves the company and takes the baby with them. The tech sector is nuts for non-disclosure agreements anyway, but add in these teaming partitions and you have really cut the kid up into parts, with no single piece carrying the potential of revealing the entire code, to put it in programming terms, or the entire child, to stay with our baby metaphor. It is not unusual (e.g., Apple) for companies to keep their engineers completely in the dark as to the product that they are developing. Engineers there take pride in learning that part of something they worked on ended up in a new device. That is protecting your intellectual property.

That model has long been used in defense industry applications, where individual engineering and science teams work anonymously to develop systems to be used in assemblies that they will likely never have the privilege to see. Hitler’s Nazis used a similar development strategy to protect the secrecy of technological developments, and to create culpable deniability for the war crimes they committed. Their extermination program was a model of agile development, which makes the DuPont connection to the history of agile development all the more intriguing. Founded in 1802, E. I. du Pont de Nemours and Company is an American chemical company. It began life as a gunpowder mill but in the 1920s they began developing polymers and a new age of plastics was born. Since then, reacting to changing needs and opportunities, they have branched into products ranging from insecticides to genetically modified foods.

Accepting that level of compartmentalization of one’s work experience is not for everybody. It creates an odd dissonance between the part one plays in a field of competing peer group members, and the behaviors that work successfully within the scrum organization that is so central to the individual agile development teams. Consisting of eight or so members, individuals in these teams are expected to engage the process fully to present ideas and analyze those of others. Agile consultants arm their client teams with processes and tools designed to create the behavior patterns for each team to mimic. Role definitions turn every person into a certain type of cog in the development process, and it places a great deal of emphasis on collaborative decision making. It is not really a place for rebels or cowboys, and in fact the process is designed to make it difficult for such types to function within the agile process.
Like many manufacturing processes, performing a role in an agile environment has some dehumanizing aspects. The only way to keep up with the project schedule is to perform a robotic sequence of events, often in a repeated fashion, until the end result is achieved.

This is, of course, a situation that has always been a defining aspect of human endeavor. We must martial ourselves together like an army of ants to accomplish anything, and ants have roles, and the only happy ant is one that accepts his condition and situation. On the other hand, “Do Androids Dream of Electric Sheep?” Science Fiction novelist Philip K. Dick posed the question in 1968. What does it mean to be human?
Agile development is designed to cut workers off from coworkers outside of their agile team, to counteract what was one of the great, but most expensive, aspects of old world work environments, which was the sense of belonging to some group that produced tangible outputs. That is not really the world that the developers of the Agile Manifesto wanted for themselves, but instead it is what they wanted for their employees. As Jon Lovitz says in those Go Daddy ads, “I don’t see you so much as a business owner, but more as a business “‘worker’”, in finger quotes.

Lovitz, or at least the character he plays in that commercial, understands one fundamental aspect of agile development: you, the individual, doesn’t matter outside of your performance of a narrowly defined role within your organization. You are a micro-managed cog in a development machine that is designed to carefully control costs while quickly moving revenue generators into the marketplace. No one development is “the complete answer”, but if visualized effectively each development may initiate the revenue generation that funds the further development of other aspects of the product.

Agile, in all of its alternating dynamics, that simultaneously optimize individual initiative while also casting agile team members as robotic functionaries, is focused first and foremost on the business’s “bottom line”. It rewards companies and those individuals who can comfortably fit lock-step into its micro-management nature. People with personalities geared to the earlier “waterfall” world do not often survive the change to agile development, and so in 2015 you have armies of older displaced workers who have been made obsolete, not by technological advances so much as by modification of our relationships with one another in the workplace.

One could suggest that we, as human beings, need to look at what is gained and what is lost by this enormous change in the way the world works. The agile world has created a structure that rewards business owners, who do not personally have anything to do with the agile process, and diminishes the place that human beings occupy in that working world that consumes most of the time of our lives. It further perpetuates the concept of human slavery, expressed through diminished personal and financial rewards.
Futurists spend a lot of time fearing “the rise of the machines”. In the form of agile development, it is really the rise of “masters of the universe” – those tech industry leaders who hatched their plan at a resort in Utah in 2001 – that are having the greatest impact on the quality of life in our world of the future, where business profits trump the very essence of how it feels to be human.