Project planning with git

Software estimation is notoriously difficult, but there are ways to make it a little easier.

For example, you can use the fact that your estimates get better as you work.

Enter version control.

One way of estimating when you will finish something is to make a "plan" file, add that file to git, and then plot statistics over time as you commit changes to your repo. For example, you might make a burn-up chart like this one:

I used this method to predict when I would complete a larger project at work, and it was eye-opening. I did not realize how easy it was for me to let project scope creep up over time, and how much it affected my motivation.

If you are interested in how well this method works in teams, do check out Allen Holub's talk "No Estimates," based on the work of Vasco Duarte. It's a great talk.

At any rate, here is my method of tracking progress using markdown and git.

A parseable plan

I write my plan in markdown because that’s what I am comfortable with, but it really does not matter what you use. As long as it is easy to parse.

# example.plan.md

1. [x] this task is marked done
2. a parent task
  - first child task
  - another child task
3. the last task

In this format, each "task" is a list item, which can itself have sub-tasks. A task is marked "done" with an [x].

I can then parse this plan into a tree, since markdown is just HTML in disguise. With this tree, I then treat the leaves as actual tasks to be completed, and the parent nodes as headings. So the number of leaves is roughly the amount of work represented in the file.

Simple enough. When the plan changes, I can track that in git.

Tracking history

Each time I update the plan, I make a new commit in git.

Since I do this a lot, I have an alias to make this as quick as possible:

alias save="git add example.plan.md; git commit -m 'save'"

The commit message is not so important, since I am just doing this to link each version with a timestamp.

Now that I have a few commits, I need to read the .git directory to pull out the relevant data.

Pulling out data from git

This was a little tricky at first, but the pygit2 module in Python was a big help. It made it easy to loop over every commit, find the plan file's source code, and save that along with the commit timestamp.

(Check out the code for more details, starting with test_git.py.)

I now have all the data I needed for the burn-up chart: timestamps and a plan versions.

Making the plot

I started out using this online plot generator from Vega-Lite. This is a great tool for quick-and-dirty plots.

But as I was copying data back and forth a lot, I ended up using an HTML template that basically did the same thing. I got it from the source code.

All I needed to do was fill in the {{ data }} placeholder in the template file and open it in a browser.

I made a little script to run the whole pipeline all at once, from reading git data to parsing the plan to displaying a plot.

./plot-plan.sh example.plan.md

This was pretty satisfying. I could plot this progress as I work, and it gave me a lot of motivation at times.

Measurement without soul

There is a trap here that I should warn you about.

When I did this kind of measurement for a while, I got into the bad habit of making more tasks than necessary, just so I could get that nice feeling of checking off tasks and seeing the plot go up.

This is bad.

This habit turns your plan into a checklist, and the meaning of each task gets watered-down very quickly. I found myself making tasks for things I really did not need to write down, and the scope of my project would just grow incessantly.

So whatever you do, do not let the excitement of measurement drive your work. Keep the plans simple enough to be meaningful, and never lose focus on the work itself (i.e. why you are doing it in the first place).

Have a try!

I wrote this post to show that tracking your progress with git is something you could do yourself, not that you should do it exactly this way.

There are a million other ways to build a progress tracker to fit your needs, and I hope you are motivated to try making one yourself.

Whatever you do, be sure to share your experience with others, even if it is just with family.

Happy coding!

~Rex