Evolving Web

Four Things to Remember When "Planning"

In 1989 I received my degree in Urban Planning. I have studied how to make plans.

In 2007 I received my certification to be part of the American Institute of Certified Planners.  I have been recognized for making good plans.

I have planned subway systems, technology rollouts, urban growth patterns, transportation infrastructure, housing needs, freeway travel patterns, business projects, software development, building houses, building buildings, and on and on.  I have planned.

I have also learned much. About reality. About complexity. About human nature. About how we plan. About politics. About blizzards. About locust plagues.

I understand, despite or perhaps because of my history, that plans are bunk, but planning is essential.  The document (the plan) we create will disappoint us the moment it is completed (if not before).  However, the act of planning, can only help us. 

Planning is knowing what goal we are trying to achieve or what problem we are trying solve (our reason for doing this), what our current workload is (our capacity, our current level of distraction, our likelihood for being derailed), who the current workload is for (how many conversations we have in flight), what that current workload is going to look like in the future, what the other options for other projects we could be doing are, what the expected size of our new project is going to be (day? week? month? Six months? - protip...the longer your project the more wrong you will be about completion times), who we would be working with, and so on.  Planning is knowing, tracking, and fully understanding the evolution of your project.

So, I promised four things to remember:

Thing 1: Your Estimate, Your Schedule, Your Gantt Chart is Always a Lie

I have built schedules for 20 years in Primavera.  I have made Microsoft Project files that would make your head spin.  Beauty in both complexity and foolishness. 

Before you start any project you need to know Hofstadter's Law and the Planning Fallacy.  

Do we want to estimate?  Absolutely.  Without some sort of guess at the amount of time something will take we have much less of an idea of its opportunity cost.  

But we need to remember that our estimate is made at the time we know the least amount about our project.  We haven't started. We've learned nothing yet.  As we begin to work, if we are tracking our work and measuring rates of completion, we can - in the future - forecast completion dates fairly accurately.  But we need to start working.

Remember...the longer the schedule, the less accurate your estimate. You schedule will always wishful thinking. Your Gantt chart is always fodder for long meetings and poor discussions.

Thing 2: Your Current Workload is Worse than You Think

If you ever take on a project without understanding your current workload, you are starting at an extreme disadvantage.  You must understand your capacity and your current workload before thinking your new plan has a chance.  You are likely already overloaded. That means you have existing relationships with customers and colleagues to complete work. Every one of those relationships takes up time and energy. The more you add, the more unpredictability you have added to your day.  Unless you are limiting your work in process and understand how much work you have on your plate (perhaps using a Personal Kanban), your new project is going to be as successful as the annual Hades Snowball Fight.

Thing 3: No Plan Survives Contact with Reality

Reality bites. I have seen projects derailed by locust plagues, acts of congress, tornadoes, blizzards, SARS, staff attrition, and...strangely enough...not understanding the complexity of our work.  We simply don't know what is truly involved in completing our work. Again, the larger the project, the more likely you will find delays due to the interactions of everything with everything else.   

Thing 4: You Forget Tons of Stuff and Remember the Rest Wrong

If you think doing something in the past makes you an expert in how it's going to work in the future, you are right.  People who haven't are just guessing when they are wrong.  You, the expert, are right about a lot of things...but you are wrong about a lot of other things.  This means your schedule will end up looking slightly better than someone who is guessing.  Rosy Retrospection and the Zeigarnik Effect combine to create one simple truth we don't remember the past accurately. We remember bad things as better than they were (which might impact both risk assessments and estimates) and we forget that a lot of work occurred at all.

State of the Planning Art

There's a lot of posts floating around the Internet about how to plan your work.  There are products to help you make lists or organize your thoughts.  That's all great, read them, take wisdom from them.  But know this: the larger the project, the larger the margin of error. If you want to plan well, create small projects.  Learn fast, adjust often.

If you want to build a real project, engage in planning. Build a system that asserts, acts, learns, and re-learns.  Build a system of honest work like this:

Goal - Have one ... know what problem you are trying to solve or value you are going to create. Know why you want to do this and whom it will help. Let this goal and the options around it evolve as you learn.

Options - Create small projects to help you achieve your goal.  Each one moves the needle a little.  The faster these are completed, the faster you get value realize your goal.  This could make it possible to achieve your goal incrementally. These are Options, make them, throw them out, use them like Lego to build value. You can only learn and adjust this way.

(Flag Waving: This is the point of the blog post. Many small adaptable batches beat big plans every day.) 

The Work - Do the actual work.

Learning - Ask yourself how the work went. Was it done in the time you thought it would be? Is it the outcome something you are professionally satisfied with? What did the customer think? What did you learn that will make the next Option you pull easier?

If this seems a lot like PDSA ... it is PDSA with a slightly better push towards small batch thinking. 

Last word

As always, if you can't describe what you are doing as a system (thanks Dr. Deming) and if you don't visualize your system, you have an assumption, not a system. 

 
  

Don’t Slow Down, Just Get It Done

My colleague and I were four days into a field exercise for an environmental impact statement or EIS. An EIS is a document every public project must complete to show what the short and long-term impacts would be if it were completed. They tend to be large and expensive. This particular project was very large and the impacts many.

Given that the project was underway and had a definite deadline, no one was willing to slow it down or change its scope. This particular task was supposed to take the two of us about a week to complete.

Four days into this particular task, we found the data we had been given was poor, the people to contact with better data were unavailable, and that the massaging of the new data we couldn’t even get would take weeks.

When we presented the managers of the project with this information, we were told Don’t Slow Down, Just Get It Done.

Get it done with what?

The data you already have.

We were then expressly told not to contact anyone else or extend this particular task.

Back then, we were completely confused as to why anyone would be so totally stupid. How could these managers knowingly ignore that we were using faulty information that would result in a useless report? These weren’t Dilbert characters, they were real live human beings.

Over the years I would watch other seemingly smart people drive their high-tech startups into the ground by insisting that an initial plan or idea go forward despite ample evidence that all that lie ahead was ruin.

What cognitive biases kicked-in to destroy these endeavors?

Irrational Escalation – This is in-for-a-penny-in-for-a-pound-ism. Once people start on a project, they find it very hard to stop – even if the project is obviously doomed or the return will not cover future costs. The managers felt that the project was already underway and even using bad data was better than delaying the project and asking for new data. The bad data (not surprisingly) later resulted in obviously flawed analysis – causing even more expensive and stressful work later in the project.

Expectation Bias – This is a world-view bias. If we believe something, we pick data points that support that expectation. Further, we expect that most data will support our expectations. We had an opportunity to create a research regimen that would yield scientific results. The managers assumed this was all a formality and that any data (good or bad) would support the original plan. This was likely underscored by “experience” that showed that most data did indeed support the original plans (which were likely suffering from expectation bias).

Planning Fallacy – A central issue here is simply that we didn’t have enough time to finish our task. The expectation bias inherent in the people who wrote the initial scope gave us only a week. This led to them, and us, falling prey to the Planning Fallacy which shows that human beings have a strong tendency to underestimate task completion times. On the ground, we saw this playing out. From the manager’s desk, the plan was correct, we were at fault.

Status Quo Bias – This bias says that once we start something, we tend to want it to continue. This is why in Collaborative Management we seek to build an explicit Culture of Continuous Improvement for teams and companies. Making change and improvement the center of the culture makes change the status quo. Needless to say, this wasn’t the case with my project back then.

The upshot here is that biases can gang up on us. Our enthusiasm, belief in ourselves, and assumption that things stay pretty much the same can combine to create seemingly logical decisions that can turn deadly. My managers no doubt reasoned that the project plan took over a year to write by well-informed professionals. There was very little cause for concern that a small part of the project might derail the whole thing.

In the end, our re-work was hardly noticed because other parts of the project became late, with much higher costs. One can only guess that the same forces were at work.

Social Media, Lean and the CIA

At the end of March, I spoke at Social Media Breakfast, Seattle (See their recap) at the Waggener Edstrom offices in Bellevue. They were live streaming the talk, but had some technical problems – so the video is in two parts. If you want to see what the audience saw on the screen, there’s also the Prezi I spoke from here.

This talk is covering the challenges that organizations have with collaboration, both internal and external. Social Media and Lean need to be applied together to provide both a set of principles and a collaborative foundation. I also talk about being a punk rocker.  So, if you want to know how punk rock, twitter and the CIA all mean the same thing … take a look!

Part 1 Part 2

Why Project Managers Fail

I’ve managed a lot of projects. I tried to count them – I can’t.  As I’ve progressed in my career, I’ve embraced a succession of methodologies that cumulatively represent a rather predictable evolution away from zero sum game management towards non-zero sum game management.

What does this mean?

As the science of project management  evolves, we are clearly learning that managing based on guess work is a recipe for consistent failure. Traditionally, we managed projects based on the trusted opinion of a project manager who estimated the length, scope, cost, and resources required for a project. These project managers have a natural tendency to underestimate, and their guesses are vetted through a process designed to save money, rather than arrive at an accurate estimate.

While well-intentioned, agile methodologies  have not appreciably changed this process. In the end, agile projects still obtain their backlogs in bulk and gather up-front estimates prone to error. For all of agile’s intentions, agile projects seldom break out of this pattern, even if they are well managed at the team level.

The top five reasons I see projects fail are (1) No slack in the system (2) Managing for the knowns (3) Not limiting work-in-progress (4) Political promises and (5) Sloppy communication.  Let’s examine these:

1. No Slack

Every system requires slack in order to work correctly.  Old reel-to-reel tape recorders needed an extra bit of tape fed into the mechanics to ensure that the tape wouldn’t rip. Roadways operate best at 65 to 70% capacity – they bog down over 80 and are at a dead stop at 100.  In mills, grain is fed into the stones at about 70% capacity – anything over that gums up the works and makes the mill grind to a halt.

Why, then, is slack so foreign to project management?  Because we negotiate it away every time.  Why do we do this?  Because we estimate at the individual task and not at the project level.

It’s like saying “Yes, my Porsche can go 185 miles an hour” and estimating a 90 mile drive at 30 minutes.  Then we’re surprised when there are other cars on the freeway.

“Contingency” funds rarely work to abate this problem because contingency is not real. It is arbitrary and has no basis in reality.  It is a lie. Without understanding our throughput, we cannot adequately estimate a project. 

Successful project managers understand how long tasks usually take, what role variability plays, and what types of tasks slow their processes down.

2. Managing for Knowns

We manage for knowns.  When we estimate, we use our history to '”know” when the project will be completed. We manage for people doing what we know they will and when they know we will.

That’s not a good idea.

Human systems are complicated. Yes, we can finish that task in 3 days provided no one gets sick, there isn’t a power outage, there isn’t a blizzard.  Well, there are colds and blackouts and blizzards.

Successful project managers worry less about making sure their Gantt chart is religiously adhered to or if their two-week iteration goes unhindered and more about how to elegantly deal with the unexpected.

3. Not Limiting WIP

Again, people and teams have a capacity. If you exceed a certain percentage of that capacity, people become stressed, distracted, and increasingly ineffective.  If a Freeway at 100% capacity is at a stand-still, its throughput is 0%.  If your people are at 100% capacity, their throughput is likewise seriously impaired.

Limiting work-in-progress (WIP) is vital for maintaining throughput. Intentionally limiting WIP frees people to maximize their effectiveness by ensuring they work at an adequate percentage of their capacity. In turn, WIP is used to limit the number of features the organization can feed a team at a time. This helps the organization appreciate the true opportunitiy costs of new features. WIP limits communicate that employees are a limited resource and helps upper management understand the true cost of labor.

Successful project managers understand the capacity of the individuals working under them and can regulate the flow of work to optimize their throughput.

4. Political Promises

Politics are stressful and are often no-win situations. Project managers are often placed in political situations that force them to promise to produce more with less resources. (“Can’t you just add this one additional feature?”) Project managers cannot win these political situations because they don’t understand their true throughput. Project managers need to concretely show true productivity statistics to weather these confrontations.

Being able to say with firm evidence, “It takes us 21 days to release one new feature”  diffuses political pressures and opens the door to real discussions.

Armed with a firm understanding of their operations, successful project managers can avoid overpromising due to ignorance. 

5. Sloppy Communication

Teams run on information. In my experience, most delays and waste can be traced back to communications. Do people know why they are doing what they are doing? Does the team know what their co-workers are doing? Are recent developments rapidly disseminated so workers feel at-ease? Are the actions of the team being communicated up the management chain? Is the value of people’s actions understood by all? Are improvements to the development system being quickly discussed and acted upon? Is waste being spotted and dealt with?

Now that we have a multitude of tools to communicate with, poor communication simply can no longer be tolerated. The increasing commoditization of goods and services leaves profit margins too slim to tolerate that waste.

Successful project managers build social systems designed to rapidly disseminate information, keep team members informed, provide a pipeline to other areas of the enterprise, and improve operations.

Closing

Agile methodologies are great, I’ve been using them since 1997 when William Rowden and I first built our TreePro product. They went a long way to solving some of these tensions. Prior to that, as an urban planner, I managed multi-million dollar planning and engineering projects.  As we move forward into the 21st century we need to understand that all knowledge work (which includes both software development and urban planning) requires a more adept type of project manager: a project manager that manages for the unknown because they understand the knowns very well – and those knowns include what were previously unknown: variability, waste, and politics.

Disclaimer: This is a blog post. It is short. Food for thought. Just think about it.

Context is Fluid – Social Media Principle #7

This post is seventh in a series of my Social Media Principles.  The base post is 10 Social Media Principles.

Principle #7 - Context is Fluid – How you view an object today will be different tomorrow.  Don’t destroy tomorrow’s value.

 

In my humble opinion, most of the business failures we’ve seen in the tech sector are due to an utter disregard for this principle.

Context is fluid. The tech you are in love with that your company is building is certainly cool.  In fact, it’s cooler than you think.

The Peril of Focus

I have seen several sites, applications and ideas become stifled because they limit their own market.  A singular focus to specific use cases has been lauded as good management. 

The reasons for this are legendary: Focusing on specific use cases avoids scope creep.  Focusing on specific use cases increases predictability and ensures a logical development cycle.

The problem here isn’t necessarily the use cases themselves, it’s the focus on a specific static context or set of contexts.

To be sure, everyone needs a plan.  Your trip home needs a route.  But you should have the flexibility to re-evaluate that route based on context. 

If your route included what used to be this bridge, would you be taking it now?

No.  Context has changed.

Your product’s context changes all the time.  People find new uses for it.  People eliminate old uses for it.  Your intended use may bore the hell out of people, but they might find another use that works great for them and could still make you rich.

Despite this, business often limits its own products’ contexts. 

How Context Works

In my 2007 post on the Seven Contexts of Human Understanding, I discuss how context radically changes based on perspective.   Geographic, political, organizational, team, personal, religious, time, etc.  Each perspective changes the context of what you might encounter. 

And context changes based on the situation, and quickly.  A baseball player at bat does very different things with 2 strikes against him than he does with none.

(See the seven contexts post for more on the workings of context.)

The Warning from Principle #7

When we create a product – any product – and narrowly define its use, we practically limit its use.  No, every product shouldn’t be able to do every thing.  But when creating your design, ask yourself “Are my assumptions limiting this product’s potential?”

Social media has been great for taking an idea devised for one purpose and re-purposing it.  Social media and Web 2.0 has also been great for building in repurposing.  The API is now mandatory – no API and you are dead.

Why?  Because everyone knows they have their own context and they don’t want to wait for you to enable it.

So, no API or a self-limiting feature set is a rocket sled ride to liquidation sale.

Technorati Tags: context,social media principles,10 social media principles,social media

Blogged at the Modus Cooperandi offices in Seattle, WA

That Guy’s An Asshole – Two Stories of Information Hoarding

This is a companion piece to my 10 Principles of Social Media series. This post illustrates Principles 2 and 4.

For quite some time, I was an urban planner.  I worked for some agencies, but for most the time I was a consultant.  I worked with all levels of government from tiny towns to the Feds. 

While that time left me with more than enough stories to fill a book, I’ll tell two of them here:

They Hate Us

The planning director for the City sat across from me.  I said, “This major road needs to have coordinated signal timing between you and that suburb.” 

“That’ll never happen,” he said, “They hate us.  Every time we change our signal timing, they change theirs just to screw us up.”

“Hmm,” I said, “Well, it’s got to happen, I’ll go over and talk to them.”

So I go to the suburb in question. 

“This major road needs to have coordinated signal timing between you and the City.” 

“Take it up with them then!” Was the reply, “We hate them! Every time we change our signal timing, they change theirs just to screw us up.”

That Guy’s An Asshole

We were creating an event management plan for a major city.  They had a huge complex of sports stadiums all grouped together.  It was possible for nearly 200,000 people to show up during concurrent events.

We had created a system to guide people to parking and to dynamically move police, roadblocks and traffic revisions based on available parking and pedestrian movements. The system would close roads, re-route traffic, and move police around.

That required communications.

“What streets have fiber?” I asked.

“We don’t have any fiber” I was told.

I was incredulous.  “what do you mean you have no fiber?”

“No fiber, has to be wireless.”

“There’s nothing with franchise agreements or other shared use?”

“No, no fiber.”

So we created a system with all wireless tech which, at the time, was much more expensive and harder to make reliable.  It greatly added to the complexity of the system.

The day we were launching, I was walking through the City offices.  I passed by a desk and something caught my eye.  I went to the guy upon whose desk it rested.

“Is that what I think it is?”

“It’s a fiber map.”

There was fiber (lots of it) down every single street.

“Can I borrow that?”

“Sure.”

I went to the project manager and put it on his desk.

“What is this?” I asked.

“It’s a fiber map.” He said.

“You told me you had no fiber.”

“Well, we didn’t want to use fiber. That guy’s an asshole.” he said referring to the person at the City responsible for the fiber. “We don’t want to deal with him.”

What This Shows Us

Even though these are examples in government, we all have stories like this from business, from school, from families.  When we stop talking and close ourselves off, only waste will result.

Technorati Tags: 10 social media principles,social media principles,social media,urban planning,obstruction,management,government

Blogged at My House due to endless snow in Seattle, WA

Stupid is as stupid shops

I have been having a love-fest the last few days. We’ve been going around talking to companies around Puget Sound.  The topic: how they are changing their management styles as we move into a clear and looming recession.

Many companies are doing very interesting and refreshing things to deepen the relationships they have with their customers.  Some of this means social media, but some of it also means better customer service and generally being more involved with the community.

In other words, as we’ve been getting closer to economic collapse – people seem to be getting more humane.

And that’s pretty awesome.  We’ll be blogging about our findings soon enough, but today I got an amusing bit in my RSS reader.  Wal*Mart didn’t seem to learn from last year’s negative press and is, again, sending out cease and desist orders for sites that are broadcasting their Black Friday sales before Wal*Mart’s official publication date.

Says TechDirt:

…l-Mart's high-priced law firm has sent a takedown notice to the site SearchAllDeals.com, which is a search engine/aggregator of various deals sites. The site doesn't host any content itself, but that didn't stop Wal-Mart from sending a false DMCA takedown claim to the site (and, of course, a false DMCA takedown is illegal). So, we have Wal-Mart, whose employees think deal sites are helpful, getting its lawyers to send out bogus takedown notices over content that isn't copyrighted, and then sending them to search engines that don't even host the content in question.

This may be interesting to watch. Local business may well have a resurgence in the next few years simply due to a few key factors:

1. You have to drive farther to Big Box stores (cf. the demise of Circuit City)

2. Big box stores tend to have large and impersonal rewards programs, vs personal service

3. Big box stores tend to use social media as exploitation tools as opposed to communication tools

4. People are plain weary of the big box aesthetic (size, parking, impersonal service, etc.).  [Apparently the big boxers get this.]

Will authenticity win out in the long run?  Let’s watch.

Technorati Tags: Wal*mart,walmart,techdirt,inside the aisle,big box,social media

What Kills Innovation

Someone once asked me, "Sure, I get that there's a niche in the market, but is there a market in the niche?"  Eight years ago, William Rowden and I started Gray Hill Solutions to create innovative software for the transportation management sector.  In the end, we were about as successful as we could be.

So I'm going to tell this story based on my perception of this particular vertical.  I believe the lessons here translate out to all verticals.  Every vertical has blind spots, cultural issues, and built-in waste.

It seemed like Gray Hill couldn't miss.  There is an entire industry called "Intelligent Transportation Systems" (ITS), but there are very few innovations in the product lines year to year.  The industry was certainly large enough to support a niche player like Gray Hill. 

The actual technologies of ITS at the time were lagging the rest of the tech community by about five years.  That led to easy identification of high-value innovation (or at least technical application) for the industry. 

Now, the industry lags behind by about 9 years and is only slipping further and further back.  What is killing innovation in transportation?  Why aren't obviously superior solutions being installed?  Why do suppliers continue to make minor improvements on feature-poor products, rather than truly innovate?

(Click on the link below to read on...)

Jim's China Tour 2007

On Sunday I leave for about three weeks in China.  I'm going for a conference and a tour of transportation technology facilities and an added seminar, but I'm really going for the food (of course).  I'll try to blog as often as I can.

Here's my itinerary:

Oct 7 - Fly out of Seattle

Oct 8 - 14 - Beijing

Oct 14- 17 - XiAn

Oct 17 - 21 - Hangzhou

Oct 21 - 24 - Shanghai

Oct 24 - Fly back home

Unfortunately, I'm fighting a cold at the moment as well, so it's going to be an interesting flight over.

If you are in any of these cities when I am, let me know!

Technorati Tags: china, intelligent transportation systems, its world congress 2007, its world congress

Grumbling for Gifts

In Seattle we are having some freeway construction.  It's fairly invasive and shuts down 3 lanes of a five lane freeway that is usually very congested at rush hour.  This means that the suburbanites can't rely on their main route into town.

These are generally the same people who vote against transit plans.

So now, they have to give up their highly subsidized means of commuting (driving) for a slightly less subsidized one (transit).  This morning, many of them drove to a park & ride to take the Sounder Commuter train.

They were very upset that there wasn't more parking for them or more trains.  On King 5 news this morning, the reporter was amazed at the number of people waiting for the Sounder commuter train.

The platform was about half full.

Meanwhile, the horrific,terrible traffic never occurred.

Why did it not materialize?

Because people actually planned their trips.  They took transit, they left later, they worked from home.  They were the solution to congestion.

Going clear back to 2001, we can see Seattle's traffic has been nearly the "worst in the nation".  This clearly indicates that this horrible wound that is crippling our economy is purely self-inflicted.  Further, it is a strong indication that the billions we are spending expanding our freeways (and not maintaining others) is wasted on people who cannot fathom that they are the problem.

We have met the commuter and he is us.

In the end, this is a networking problem, a social inertia problem.  Up until this morning, the assumption was that driving was the only option to get downtown that was socially acceptable.  But suddenly, there was an issue so big, so large, that people couldn't fathom continuing their usual routine - even though their usual routine wasn't working in the first place.

These commuters were already stuck in traffic for up to an hour.  These commuters already had alternatives available to them.  They already had the pain and the remedy.

But as a network, as an organism, the culture of suburban Seattle couldn't fathom taking transit, altering their commute times, or having flexible schedules.

Why?  Because the freeway itself is seen as an immutable object in suburban culture.  The freeway itself, according to suburban lore, is always under the control of the driver.  Congestion is other people's fault and interferes with the driver's freedom.  But the driver is never personally part of that congestion.

It often takes an outside disruptive influence to alter the behavior of a network.  In this case, WSDOT was the outside influence that disrupted this world view.  WSDOT told them point-blank, "Your beloved I-5 will shrink dramatically for two weeks."  This disruption was so profound that it altered their behavior, even though the status quo should have already warranted this.

Tomorrow, we shall see if people want to game the system by commuting tomorrow because volumes today were so low.  After that, we shall see if they learn any lessons whatsoever from this experience.

Technorati Tags: suburb, suburbs, suburban, freeways, congestion, construction, paradigm shifts, traffic, urban planning, pogo, walt kelly, community indicators, community