Signs That You’ve Outgrown Github, Part 3: Merges

This is part 3 in a series about the limitations of Github, and how they might be holding your team back from its full potential. If you want to go back and catch up, here’s Part 1 and Part 2.

Go ahead, I’ll wait right here.

Done? Okay, let’s move on.

Today’s discussion is about getting code merged. It’s one of the most fundamental things any software team has to do. If your team is anything like ours, it’s something you do many times every day. It might even be the kickoff for your Continuous Integration/Delivery process, as it is for us.

Reliable merging is the key to getting multiple things done at once

Getting things out quickly is important, but preventing outages from bad code is even more important. You probably want an easy way to review and even test code that doesn’t distract too much from the work you’re already doing.

Once again, Github has a wonderful system for promoting code from one repository or branch to another — the Pull Request (PR herein). In particular, the PR is great for open-source projects where people might want to contribute, even if they’re not members of the main project. The proposed contributions can be examined by the main project member(s) and be pulled in only if they’re helpful.

Typical Github pull request — you can see the code changes and their purpose, all in one place

But like many other Github features, you may find the PR process to be mis-aligned to your needs, in a way that creates a little extra delay and a bit of confusion every time you use it.

Pull Requests 101

For those who haven’t tried one yet, a pull request (PR) is a special kind of task, asking someone to merge a set of changes (commits) from one place to another. In more general terms, you can think of it as promoting a chunk of work from one level to another — such as from a development branch to test and then to production, or from a hotfix branch to long-term-support.

A simple example of promotion — taking a specific change from one environment to another

Because it’s a request, it doesn’t involve any access to the project repository from any non-members. The project team can review the proposed changes and take them, ask for revisions, or ignore them. It’s a great model for open-source or otherwise loosely-coupled groups to share ideas and improvements.

Keep It In One Place

But that flexibility comes at a cost. Pull Requests are opened and managed separately from individual tasks, so you’re basically creating another task to review each task’s work. The open or closed status of each task can be independent of the status of the related PR. Additionally, there’s nothing stopping someone from submitting changes for multiple tasks in one PR, which can be confusing and difficult to review.

For software teams that aren’t open-source, this loose coupling actually creates more process, more overhead, and time thinking that could be spent doing instead.

Ask yourself — wouldn’t it be a lot easier to merge the code as an integral part of the task itself?

Singularity Of Purpose

Let’s start with an assumption — that in order to change your code, there should be a defined reason for doing so.

You’re writing code because something needs to be added or fixed. There’s a use case. A story. A bug. A feature. A customer who wants something. There’s a reason to change what you already have.

You probably also want to do two things with your tasks:

  1. You want to create a plan ahead of time, for which things will be done, by whom, in what order.
  2. You want to keep track of progress as things move along.

Once you start depending on tasks for planning and tracking, you can begin to improve your overall process, reducing the number of steps and the distance between idea and working code. As you do, separate PRs may start to lose their appeal. Asking developers to open a separate kind of ticket to get code merged is a hassle. Allowing them to put multiple bug fixes into one merge is asking for confusion and mistakes.

If you’re delivering code using a well-defined workflow, PRs can actually cause problems:

  • Audit trail — It’s difficult (or impossible) to know later which code changes went with which task.
  • Larger merges — the code review itself becomes much more complicated, since there are more commits, more changes files.
  • All or nothing — If you like the changes for task #1, and for task #2, but there are problems with the tests for task #3, the whole PR is sent back for rework. This means you’re likely sitting on changes for longer.
  • More conflicts — Pretty simple math: (Larger merges) + (All or nothing) = More conflicts.

Since there’s no way in Github to limit the content of a PR, there’s no good way to prevent this kind of behavior. Creating a PR for every single bug becomes a tedious time-sink that doesn’t add value to your day.

Now, you might argue that a Github PR can act as the task itself, and it does — but not really. PRs are only retrospective, meaning that you create one after (or while) doing the work. If you don’t create tasks before doing the work, then you’ll never have any way of planning or tracking progress.

Simplify, Simplify

For most teams, the overlap between tasks and PRs is additional work that doesn’t generate any value. What you really need is a way to automatically detect code changes, review those changes and then promote them to dev, test and production, all as part of the task.

This kind of integration means that you can go back to the task later, understand the intent of the change, and also see the code changes that went with it. Your task tracking becomes a source of institutional memory, so that people can move in and out of the team, or across different features without making old mistakes over and over again.

If your tools are preventing you from improving your process, maybe it’s time to improve your tools.

Come try GForge Next for simple, comprehensive and elegant collaboration.

Signs That You’ve Outgrown Github, Part 2: Task Management

In my last post, I introduced a set of features (and failings) that might have you wondering if Github can grow with your team. In this post, I’m talking about tasks, workflow and keeping things moving.

Github’s Workflow Is Simple And Cool

When you’re starting out, Github’s approach is really helpful. You can create a task with just a summary sentence. If you want, you can add a more detailed description and some labels.

Tasks are open or they’re closed, and you can close a task with a commit. From a developer’s perspective, it’s wonderful to grab a task and write some code, then update the task without having to visit the website. You get to stay in your IDE, your command-line — whatever tools you’re using to do the work.

The Real World Gets Complicated

These features work very well for teams that are just starting out, and for projects that may have somewhat disconnected participants (e.g., open-source projects). But as your team begins to deal with marketing, sales, customers, or even other dev teams, you may rub up against some important limitations.

There’s more to life(cycle) than Open and Closed

Very often, there are other pieces to your workflow than just coding. There could be planning, design, code review, testing, deployment, and other activities. It’s pretty common to reflect some of that workflow in the status, so that everyone can tell where a task is and what needs to be done next, without having to re-read then entire history.

Closing a task just because you’ve committed some code assumes that everything else will go right. Usually, committing code is the beginning of the process, not the end.

Not everything fits in the same box

Not everything is a task to be done in the code. Sometimes, you want a place to flesh out a user story, or discuss the design for a specific page. Maybe you need to track servers, or which customers have which version of your product.

You might be able to jam these different kinds of information into a Github task, but it ends up being a bigger version of the labeling problem from my previous posting.

What’s going on?

Github has some pretty decent release/milestone tools, and you can group tasks into a milestone to track overall completion. That project-level view can be great, but what about planning and tracking your current sprint? Or planning and tracking your own day?

Standups get a lot easier when you can automatically see what the team worked on recently, and when the team can identify what they plan to work on.

What’s really important?

This is one that everyone runs up against pretty quickly. Putting a priority on each task allows the team to keep moving, by knowing which tasks to work next. Without prioritization, everyone’s just guessing about what is most important.

Github has no way to prioritize tasks. Even if you use a label (to beat a dead horse), you won’t be able to sort them.

Plans get tangled up

It’s pretty easy to enter and manage tasks in Github, and depending on your level of detail one task might be completely stand-alone from another. Almost inevitably, though, a task will be blocked, waiting for some other task to finish. Your new web page needs an API update from someone else. A customer’s support ticket might be waiting on a bug that someone is working on. Or a use case can’t be complete until all three web pages are working together.

The ability to manage dependencies has been asked and discussed many times over the years, and Github just doesn’t want to take on the complexity. It doesn’t have to be a Gantt chart, but making sure that everyone knows when tasks are blocked (and when they become unblocked) is key to maintaining project momentum.

A More Realistic Approach

First off, don’t expect Github to change. The simplicity of their task management is perfectly suited for many, MANY projects. It’s easy to get started with, and is a great place for sharing open-source and personal projects.

But maybe your team has started to suffer from that simplicity, and you’re looking for something to fill in the gaps that Github leaves. You might even be tempted to try one of the dozens of “bolt-on” tools that claim to integrate tightly with Github.

Instead, let me suggest that what you really need is something that was built from the ground up to start simple, grow to be comprehensive, and stay elegant. Something that’s easy to use, but allows more complexity as you grow into it.

Here are a few more simple pieces of advice:

  • Instead of shying away from adding different status values, be honest about the workflow you want and make the tool serve your needs.
  • Consider that maybe you need a different set of fields for tracking use cases or UI designs, versus coding tasks, versus customer support.
  • Don’t settle for something that’s too simplistic, or a patchwork of loosely-coupled tools, when working around their limitations will cost you time that’s better spent on your actual mission.

If your tools are preventing you from improving your process, maybe it’s time to improve your tools.

Come back soon for another sign that you might have outgrown Github .

Come try GForge Next for simple, comprehensive and elegant collaboration.

Signs That You’ve Outgrown Github

Github is WONDERFUL. I have used it for a long time now, and it’s great. It’s easy to get started, and especially for one-person projects, it really helps keep things organized.

But as you start to add team members, competing priorities, and overall complexity, something changes. In lots of little ways, Github starts to show that it just wasn’t designed for long-haul software projects, with many team members and a non-trivial workflow.

Here are some of the things you might already be dealing with:

  • You’ve created so many labels that you’ve written a wiki page to tell everyone how to use them (and how NOT to use them!)
  • Maybe you want to keep track of bugs and support tickets separately.
  • Perhaps you don’t think that developers should be able to close a ticket just by committing code.
  • Maybe you’re tired of having to search in Google Docs, Basecamp, Slack and Github to find where you talked about that new feature.
  • Could be that you need to manage the relationships between tasks that depend on or block each other.

I’ll do a blog post for each of these topics. This week, let’s start with the first one — labels.

Labels

Labels are quick and easy to use, offer a lot of flexibility and require virtually no setup or configuration overhead. The default labels in Github look something like this:

Those are probably fine for 98% of new Github projects. Over time, however, most teams will expand on the list of tags to include different values than the default list, and even different types of data. Adding categories for basics like type, priority, and status makes a lot of sense when you’re managing a backlog. But all of those new values expands the list beyond what you can keep straight in your head.

So, how can teams use complex label sets reliably? One common convention is to use a prefix word to identify the group to which each label belongs (courtesy of Dave Lunny):

This approach has the benefit of including the type of label as part of the label text, so you won’t be confused between “Status: Test”, “Environment: Test”, and “Task Type: Test”. Visually, the various colors can be used to show differences in the specific values, like production vs non-production, or high priority vs lower priority.

The downside is that when viewed across label types, the different colors are confusing and even downright jarring. Instead of quick visual signals based on color, you actually have to read each label value. It’s bad enough when looking at a single task, but when browsing a list of tasks, it can be a hot mess.

Another approach is to use similar colors for grouping. Here’s an example (from robinpowered.com):

While it’s much easier on the eyes, you no longer have the label type included with the label value. Also, this particular set of labels combines and separates a lot of values in ways you won’t appreciate later — for example, why is the production label in “Problems” and not “Environment”? There’s likely information on a wiki page somewhere, explaining that you don’t choose anything from “Environment” when it’s a production issue, right?

This next example isn’t really an example at all, but a cautionary tale. Imagine walking into your first day with these guys:

This isn’t someone’s drug-addled fantasty project, either — it’s servo, a very active and popular project on Github.

Tools For Your Tools

In response to this widespread need, there are actually tools (like git-labelmaker) dedicated to helping Github users automate the creation and management of complex sets of labels, using different naming conventions, color schemes and labeling styles. Which is very cool — except that it seems like an awful lot of work to go through, doesn’t it? If someone has created a tool to help you use another tool, maybe it’s the wrong tool.

Tag/label functionality was never designed for complex values or workflow.

A Right-Sized Solution

Instead of a mashup of various labels, organized by a prefix word or a color scheme, consider this layout (with a few additions that are, IMO, obvious):

Separating these various concerns also allows you the flexibility to require some fields when a task is created, and wait for the right step in your workflow for others. Instead of manually enforcing your task management process, you can allow your tool to do it for you.

One last benefit — if you’re using separate fields instead of specific tag values, then changing the name of the field or the values over time won’t break old data. The new field name or value(s) will be reflected in older tasks automatically.

If your tools are preventing you from improving your process, maybe it’s time to improve your tools.

Come back soon for another sign that you might have outgrown Github — next time, we’ll talk about Github’s one-dimensional task tracking.

Come try GForge Next for simple, comprehensive and elegant collaboration.

GForge v6.4.3 Released

Today we are happy to announce the immediate availability of GForge Advanced Server v6.4.3.  Other than being a bug fix release, this will likely be our last update to the 6.x series as we intend to ship GForgeNEXT later this year.

Important note: For anyone using GForge under the default license, which lets you run GForge AS free with up to 15 users without support, this number has been reduced to 5 users. Contact us at feedback@gforgegroup.com if you have any questions about this change.

Download GForge Advanced Server v6.4.3 Now!

New Features in GForge v6.4.3

  • Docman – Prior to this release, when browsing files in docman the link to the file pointed to the last version which had the version hardcoded as part of the URL. We now have use a URL that removes this hardcoded reference which means if you share the URL to the current version you will always get the latest version regardless of how many new versions have been added.  You can still access the direct link to old version through the right-click menu.
  • Project Export/Import – There were a number of issues preventing the export of a project from being imported into another GForge instance. You can now port projects between GForge instances provided both are running 6.4.3.
  • Licensing – Site administrators will receive warnings when they are approaching the maximum number of users allowed on their license.  Additionally we have given site administrators the ability to deactivate users in the event the hit that limit. Prior to this release when the system exceeded the maximum number of licensed use the system would give all users an error message.

The Gory Details

GForge v6.4.3 includes a number of bug fixes and security updates. For the full details of this release, please have a look at the Complete GForge v6.4.3 ChangeLog

Download GForge Advanced Server v6.4.3 Now!

How Many Project Tools Do You Really Need?

In my last post, I talked about some criteria for choosing between all-in-one or best-in-breed tools. Now, I’ll apply those criteria to our product space, and try to show why a unified team collaboration tool makes more sense than what you might be living with now.

So, let’s imagine that you’re looking to solve the following problem: How can we get everyone rowing in the same direction?

By “everyone,” we mean executives, product owner(s), designers, development staff, customer service, maybe even customers themselves.

And by “rowing,” we’re talking about all of the things it takes to get your product to the customer — things like planning, design, coding, marketing, testing and support.

The Accidental Integrator

The worst-case scenario is also the most common — each discipline area adopts whatever tools appeal to them.

NB: This is a simplified version of actual events from a previous job. The details have not been exaggerated, and the people involved were otherwise very capable, smart, and successful.

  • The Product Owners all decide to use Trello, and thus force it on the development staff.
  • The designers prefer Basecamp, where they can organize screenshots for the various user stories they’re working on. This means that the Product Owner has to look there as well, and so do the developers, and possibly also the customers.
  • The developers decide on Github, since it’s easy to start and Git is pretty awesome.
  • Customer Support starts a wiki, so they can get all of their procedures and help scripts in one place.
  • They already use FogBugz for tracking customer issues, and neither Trello nor Github has the features they need. This means that the Product Owner and developers have to look in FogBugz almost every day, as they respond to problems reported by customers.
  • The executive staff needs all of the above to keep a handle on what’s happening where. They also create and manage a couple of spreadsheets in Google Drive, to keep summary data for the various projects that are in process, and to prioritize upcoming work.

Here’s what that set of tools ends up looking like:

Pretty, pretty arrows.

Of course, some of these tools can be tied together with vendor-supplied integrations, web hooks, or some simple scripting. Adding these integration points might make things more convenient for users, but it doesn’t change the overall picture:

Now with 20% more arrows!

Inevitably, some amount of developer time each month (or week) will now be spent troubleshooting why x is no longer taking to y, and fixing it. People also regularly forget which password goes to which system, and need resets on a recurring basis. [Ed. This happens to me ALL THE TIME.]

Regardless of how good those integrations are, everyone still has to search across two or three or more tools to find what they’re looking for.

Each tool you add is another silo of data and process, and more friction in the overall workflow.

A (Kinda) Integrated Solution

Many teams opt for the Atlassian/Microsoft/IBM approach — buying a set of tools from the same company, with the promise of tighter integration. It’s sold as an all-in-one solution, in as much as you’re buying from one vendor. And for the most part, that’s true — until it’s not.

Tool suites like these do a much better job at tying related data together — build status can show up in the related tasks, support tickets can reference wiki pages, chat can become your company’s heartbeat. All of those capabilities help to decrease friction, distribute knowledge, and keep the entire organization plugged in to the right priorities.

Oh, that’s much simpler. (source: ClearVision)

But they’re still separate tools. You still have to flip from one app to another to search, get the details, or make updates. Each tool is still deployed and managed in separate software versions, which (sometimes) fall out of compatibility with each other. And don’t forget that you’re still going to pay licensing and maintenance for each tool. Even at a bundled rate, you’re probably getting soaked.

Half-way integrated is better than the patchwork quilt approach, but it still creates implicit silos of data, and introduces other hidden costs. It’s like being half-way delicious.

The Whole Enchilada

Speaking of delicious, what happens when you *actually* integrate tasks, wiki, chat, releases, documents, source code, support, and continuous delivery? If you could get all of these chunks of functionality in a single system, how would it affect your experience and productivity?

Isn’t that so much simpler?

There are some big benefits:

  • A seamless experience for managers, development staff, creatives, support and even customers to get involved in.
  • A single source of truth for what’s important, what’s done, and what’s next.
  • One place for your institutional knowledge to live on. On-boarding becomes easy. Hard-won company wisdom doesn’t disappear when someone leaves or changes roles.
  • One search box that is deeply connected to all of your projects, source code, documents, information, tasks and discussions. Find a specific key word or error code no matter where it’s hiding.

There are lots of situations where best-of-breed thinking makes sense. In areas where a specific tool improves your productivity, saves money, or just makes work more fun, it can be worth it. If the need you’re filling isn’t central to your business, then it’s not a huge risk to try (and maybe abandon) something new. But that’s not what we’re talking about here.

The way your team organizes and works together is possibly the most important and valuable part of your entire business. Trying to run things using a patchwork quilt of point solutions might seem cool at first, but it’s probably not where you want your team spending their time.

That’s why we built GForge. Because we do what you do every day, and we think it should be easier.

Come try GForge Next, and let us know what you think.

Note: My editor wanted me to include email as part of this discussion, but it added way too much to the article. GForge does some great things with email, and I will publish something soon to show how and why.

The Same Old, Brand-New Argument: All-In-One or Best-Of-Breed?

Introduction

For at least the past twenty years, IT folks have been faced with the same basic problem when choosing whether and which products to adopt. The term “Best of Breed” is commonly used to talk about products that offer high quality in a narrow set of features. “All in One” products, on the other hand, combine features that may cross business process or technical boundaries.

Different vendors have very different views on whether customers need a bunch of features integrated in a single product, or whether they would rather manage a set for more focused products that they (or someone else) can stitch together.

What’s the right way to go? The answer is, of course, “it depends” — and while this may be accurate, it’s not very helpful without some practical criteria to apply.

The Trade-Offs

Learn To Let Go

First things first — as a customer, buying someone else’s product means buying into their approach, their decisions, and their limitations. You’re almost certainly not going to get *exactly* what you want. Then again, you won’t be building and maintaining your own code, spending money on adding features, and your development staff can be off chasing your *real* business goals.

Look at the Big Picture

Another big trade-off involves the *rest* of your business technology. How does your current need fit in with the rest of your business systems? If you can cover more business requirements with one product, it means fewer integration points. If your requirements are very complex or specific, it might be worth the extra dependency to go after a best-of-breed tool in a given area.

Watch the Uptime

Lastly, consider outages and support. It’s inevitable that you’ll depend on your vendor(s) to come through for you when something goes wrong. Make sure you have a well-defined level of service with each vendor.

Outages can and will occur, and they affect your business continuity. Having a set of smaller, independent services from different vendors might seem like a good hedge against downtime, but in reality more moving parts *always* means a higher chance of failure. If four out of five systems are up, it doesn’t mean you have 80% functionality — interdependency usually drives that number down pretty quickly as you add components.

What’s Right for You?

With those trade-offs in mind, let’s go through a few questions that you can apply to your own situation, to help identify where you might find the most value:

1. How Big (Small) Is Your Scope?

If your needs are pretty narrow (e.g., file storage, web analytics, payment processing), then it’s likely to be well-covered by a best-in-breed solution. If you need lots of features, or a lot of complexity within them (think workflow, document management, billing or accounting), then an integrated option will offer less difficulty to get up and running, even if it has some limitations you don’t love.

2. How Much DIY Can You Handle?

This one is pretty simple — the more pieces you add to your quilt, the more stitching you’ll need to do. For example, you’ll need to keep your list of customers updated between CRM and project management, or maybe get build status in your work chat.

Nowadays, it’s very typical for applications to offer an API right out of the box. It’s also pretty common to have some integrations baked into tools — fill in a field or two, check the “Enabled” box and it’s connected. But the ease of initial adoption can misrepresent the ongoing costs to keep things the way you want them. Here are some examples:

  • Documented APIs are typically stable and reliable, but your custom integration with an API can become fragile over time, as your needs become more complex.
  • Built-in integrations provided between vendors (especially “web hook” type integrations) are always vulnerable to compatibility issues between the vendors, as they add (and retire) features over time.
  • Troubleshooting problems between multiple vendors is not for the faint-of-heart — you will often find yourself stuck in the middle, trying to prove that you have a problem they can solve.

If you’re a completely bootstrapped startup, where you have more time than money, it might make sense to invest that time into getting the tools you want tied together. As your organization grows, however, the balance between time and money often changes, and you’ll need to re-evaluate some of those early decisions.

3. Who Are You Getting Involved With?

Regardless of which way you go, you’ll want to know some things about your vendor(s) before you sign on. Here are a few starters:

  • First and foremost, are they going to disappear one random evening, with all your data?
  • How long have they been around?
  • How do they deal with customers during the sales cycle? The support cycle?
  • Do they solicit/accept/ignore requests from their customer base? Are they responsive to requests?
  • What levels of support (free or paid) are available? Do they promise a specific level of service?
  • Do you know anyone else who is a customer? What do they think?

Depending on your size and level of formality, these questions may become much more important. Newer, smaller, bootstrapped companies may care a lot more about what they can get now, and less about who’s answering the phone at 3AM. Organizations that have to answer to customers, boards of directors, investors or regulatory authorities might have an entirely different view. Uptime becomes much more important once you have paying customers, and people relying on your services.

4. What Will You Need Next Year? In Three Years?

If you’re a new company, patronizing another new company can seem like a great idea. Finding a focused vendor to partner and grow with can be a great fringe benefit — unless they go out of business or pivot away from what you need. Regardless of how good a relationship is at the beginning, it’s important to keep in mind how you’ll get out if and when it’s time to move on.

Some services are easier to change than others, like payment processing or CDN — you can even use two vendors concurrently and make a soft switchover. For other tools, like bug tracking, CRM, or internal tooling (e.g., database, message queue, web platform), changing vendors can take time, attention and planning away from your more strategic goals. All of those distractions cost opportunities, sales, and revenue.

But that’s not even the worst-case scenario. Instead, many teams will continue using tools that don’t support them strategically, struggling along with more and more string and duct tape around a core that is no longer suited to them. This is a quiet, passive killer of your team’s momentum and ability to innovate — especially if certain tools or systems become off-limits for discussions about improvement.

In general, I try to buy software and services the way that parents buy clothes for their kids. Sure, they’re a little too big at first, but if you choose wisely, you’ll find something you can grow into. Maybe even something you never outgrow.

If you’re looking for a task/code/team collaboration tool that you’ll never outgrow, come check out GForge Next: https://next.gforge.com

The Weekly Forge: 10/07/2016

The biggest development this past week is work on GForge AS v6.4.3 has started and we anticipate having it available for current customers by the end of this year.  Obviously we’ll continue work on GForgeNEXT which is accurately reflected in the highlight from last week below:

  1. Git Server Side Merges – Right now server side merges work between any two branches except when the destination branch is master. Turns out the is a bug with Git itself and we already have a patch from the Git development team and we’ll continue testing that out.
  2. Continuous Integration – We’ve begun designing the mocks required for building a Continuous Integration dashboard for all GForgeNEXT projects using Jenkins. This will complete our vision of making the integration between GForgeNEXT and CI systems tighter which reflects how integral they are to software teams.
  3. GForgeNEXT iOS – The first view for the GForgeNEXT iOS app has been stubbed out and work will continue over the coming months.
  4. Dependency Management – When a ticket has predecessors (i.e. blocking dependents) when those tickets are taken care of GForgeNEXT will notify the assignee and submitter of the ticket that work can begin on that ticket.  Those notifications will happen with growls which are also sent via push notifications via Adroid/iOS application. This is a big step towards helping PMs and project leads with automated dependency management.
  5. Improved Link Handling in Docman – Currently in 6.4.2 and prior the links to documents in docman explicitly ping to the most current version (i.e. version 7). The problem is if you email that or include it in a ticket or wiki entry that version may eventually be replaced with newer ones.  We’ve made it so the links in docman will always give users the most current version.  Don’t worry you can still explicitly send the link to a specific version of a file if you want.
  6. A handful of bug fixes for v6.4.3 are being actively worked on.