GForge v18.1 Released!

Just a little over a month ago we ushered a completely revamped GForge platform dubbed GForgeNext and today we are happy to announce the release of v18.1.  Please remember we have changed our version numbering to reflect the year and the number of the release. Since this is the second release of 2018 this version coincides to v18.1 which should help customers quickly know how many versions behind they may be.

Take a tour of GForgeNext!

Getting Started with GForgeNext

The biggest change in 18.1 is the addition of SVN commit hooks. This means that all customers using both Git and SVN can safely upgrade to this version. For our remaining customers still using CVS we will be adding that support in v19.0 due out the first quarter of next year.

  • The v18.1 ChangeLog will help you understand the changes you can expect.
  • The GForgeNext FAQ will answer most of your questions but don’t hesitate to send additional questions.
  • We are still encouraging customers to reach out to us for a free consultation on the planning and upgrade process. If we don’t hear form you we will be reaching out to all our customers over the coming week.

Download GForge v18.1 Now!

Signs That You’ve Outgrown Github, Part 3: Merges

This is part 3 in a series about the limitations of Github, and how they might be holding your team back from its full potential. If you want to go back and catch up, here’s Part 1 and Part 2.

Go ahead, I’ll wait right here.

Done? Okay, let’s move on.

Today’s discussion is about getting code merged. It’s one of the most fundamental things any software team has to do. If your team is anything like ours, it’s something you do many times every day. It might even be the kickoff for your Continuous Integration/Delivery process, as it is for us.

Reliable merging is the key to getting multiple things done at once

Getting things out quickly is important, but preventing outages from bad code is even more important. You probably want an easy way to review and even test code that doesn’t distract too much from the work you’re already doing.

Once again, Github has a wonderful system for promoting code from one repository or branch to another — the Pull Request (PR herein). In particular, the PR is great for open-source projects where people might want to contribute, even if they’re not members of the main project. The proposed contributions can be examined by the main project member(s) and be pulled in only if they’re helpful.

Typical Github pull request — you can see the code changes and their purpose, all in one place

But like many other Github features, you may find the PR process to be mis-aligned to your needs, in a way that creates a little extra delay and a bit of confusion every time you use it.

Pull Requests 101

For those who haven’t tried one yet, a pull request (PR) is a special kind of task, asking someone to merge a set of changes (commits) from one place to another. In more general terms, you can think of it as promoting a chunk of work from one level to another — such as from a development branch to test and then to production, or from a hotfix branch to long-term-support.

A simple example of promotion — taking a specific change from one environment to another

Because it’s a request, it doesn’t involve any access to the project repository from any non-members. The project team can review the proposed changes and take them, ask for revisions, or ignore them. It’s a great model for open-source or otherwise loosely-coupled groups to share ideas and improvements.

Keep It In One Place

But that flexibility comes at a cost. Pull Requests are opened and managed separately from individual tasks, so you’re basically creating another task to review each task’s work. The open or closed status of each task can be independent of the status of the related PR. Additionally, there’s nothing stopping someone from submitting changes for multiple tasks in one PR, which can be confusing and difficult to review.

For software teams that aren’t open-source, this loose coupling actually creates more process, more overhead, and time thinking that could be spent doing instead.

Ask yourself — wouldn’t it be a lot easier to merge the code as an integral part of the task itself?

Singularity Of Purpose

Let’s start with an assumption — that in order to change your code, there should be a defined reason for doing so.

You’re writing code because something needs to be added or fixed. There’s a use case. A story. A bug. A feature. A customer who wants something. There’s a reason to change what you already have.

You probably also want to do two things with your tasks:

  1. You want to create a plan ahead of time, for which things will be done, by whom, in what order.
  2. You want to keep track of progress as things move along.

Once you start depending on tasks for planning and tracking, you can begin to improve your overall process, reducing the number of steps and the distance between idea and working code. As you do, separate PRs may start to lose their appeal. Asking developers to open a separate kind of ticket to get code merged is a hassle. Allowing them to put multiple bug fixes into one merge is asking for confusion and mistakes.

If you’re delivering code using a well-defined workflow, PRs can actually cause problems:

  • Audit trail — It’s difficult (or impossible) to know later which code changes went with which task.
  • Larger merges — the code review itself becomes much more complicated, since there are more commits, more changes files.
  • All or nothing — If you like the changes for task #1, and for task #2, but there are problems with the tests for task #3, the whole PR is sent back for rework. This means you’re likely sitting on changes for longer.
  • More conflicts — Pretty simple math: (Larger merges) + (All or nothing) = More conflicts.

Since there’s no way in Github to limit the content of a PR, there’s no good way to prevent this kind of behavior. Creating a PR for every single bug becomes a tedious time-sink that doesn’t add value to your day.

Now, you might argue that a Github PR can act as the task itself, and it does — but not really. PRs are only retrospective, meaning that you create one after (or while) doing the work. If you don’t create tasks before doing the work, then you’ll never have any way of planning or tracking progress.

Simplify, Simplify

For most teams, the overlap between tasks and PRs is additional work that doesn’t generate any value. What you really need is a way to automatically detect code changes, review those changes and then promote them to dev, test and production, all as part of the task.

This kind of integration means that you can go back to the task later, understand the intent of the change, and also see the code changes that went with it. Your task tracking becomes a source of institutional memory, so that people can move in and out of the team, or across different features without making old mistakes over and over again.

If your tools are preventing you from improving your process, maybe it’s time to improve your tools.

Come try GForge Next for simple, comprehensive and elegant collaboration.

git support in GForge

GForge provides full support for the git SCM (Source Code Management) tool, in addition to SVN and CVS.

Unlike SVN and CVS, git is a distributed tool where each user has a copy of the full repository in his or her machine. The basic workflow of git involves committing changes against the local repository and sharing the changes via pull and push operations.

Since commits in the local repositories cannot be “seen” by the GForge server, GForge sets up a global, centralized (“bare”) repository where the changes can be pushed to.

Setting up your project for git

To enable git support for your project, you need to be logged in as a project administrator. Get into the “Admin” section of the project, and select “GIT” as the SCM tool. Click the “Submit button”.

This will create a bare git repository in the GForge server. This may take up to 15 minutes, you can check if the repository was created by clicking on the “Git” section in the project. If you get the message “The SCM repository for this project hasn’t been created yet”, you need to wait a couple of minutes.

Once the repository is created, you will see the empty repository in the Git section:

Adding the initial data

At this point, the git repository is empty and is not very useful, so some data must be imported first. You will have to push an existing local git repository into the GForge repository.

For example, let’s assume you have an existing repository with source code in HOME/myrepo:

$ ls -a ~/myrepo
. .. bin conf cronjobs .git languages lib README.txt wwwlib

This is a working directory with the source code and the git repository is stored under the .git subdirectory.

Now, if you want to import this repository into the GForge project, you need to use the git push command, but first, you need to find out what’s the repository location (i.e. what’s the full URI for the repository in the GForge server). In the project, click on the Git section and then click on the Access info page. This wil give you the URI for the repository:

Note that the URI that’s used to clone the repository is the same you’ll use to push the changes.

So, with this information in hand, you can now push your repository to the GForge server:

~/myrepo$ git push ssh://<username>@<gforge_server>/gitroot/gitproject1 master
<username>@<gforge5.vm>'s password: <your GForge password here>
Counting objects: 113, done.
Compressing objects: 100% (104/104), done.
Writing objects: 100% (113/113), 86.39 KiB, done.
Total 113 (delta 51), reused 0 (delta 0)
To ssh://<username>@<gforge_server>/gitroot/gitproject1
* [new branch] master -> master

Note that this will only push the master branch to the repository.

The changes you’ve pushed will now be visible when you browse the repository:

Working with the git repository

Once the repository has some code, users can start working with it. For this, they must clone the repository from the GForge server. This is done with the git clone command as seen on the “Access info” page:

$ git clone ssh://<username>@<gforge_server>/gitroot/gitproject1
Initialized empty Git repository in /home/marcelo/gitproject1/.git/
<username>@<gforge_server>'s password: <your GForge password here>
remote: Counting objects: 113, done.
remote: Compressing objects: 100% (53/53), done.
remote: Total 113 (delta 51), reused 113 (delta 51)
Receiving objects: 100% (113/113), 85.51 KiB, done.
Resolving deltas: 100% (51/51), done.

This will create a directory with the project’s name (in this case gitproject1) with a local copy of the full repository.

The basic workflow with git involves 4 steps:

  1. Pulling changes from the GForge repository and updating the code
  2. Changing the source code
  3. Committing changes against the local repository
  4. Pushing changes to the GForge repository

For example, suppose someone has changed a files in their local repository and pushed them to the GForge server. The following describes a typical git session:

~/gitproject1$ git pull
remote: Counting objects: 7, done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 4 (delta 3), reused 0 (delta 0)
Unpacking objects: 100% (4/4), done.
From ssh://<username>@<gforge_server>/gitroot/gitproject1
70aa173..9450e42 master -> origin/master
Updating 70aa173..9450e42
Fast forward
wwwlib/index.php | 1 +
1 files changed, 1 insertions(+), 0 deletions(-)
~/gitproject1$ vi lib/ScmgitPlugin.php
(... edit, edit, edit...)
~/gitproject1$ git commit -a -m "Fixed bug [#16]"
Created commit d1a4acb: Fixed bug [#16]
1 files changed, 1 insertions(+), 0 deletions(-)
~/gitproject1$ vi cronjobs/create_git.php
(... edit, edit, edit...)
~/gitproject1$ git commit -a -m "Fixed another bug [#17]"
Created commit b8b281d: Fixed another bug [#17]
1 files changed, 4 insertions(+), 2 deletions(-)
~/gitproject1$ git push origin master
Counting objects: 13, done.
Compressing objects: 100% (8/8), done.
Writing objects: 100% (8/8), 777 bytes, done.
Total 8 (delta 5), reused 0 (delta 0)
To ssh://<username>@<gforge_server>/gitroot/gitproject1
9450e42..b8b281d master -> master

Note that when pushing, you usually don’t have to specify the full URI of the repository since it assigns the alias “origin” to the repository you cloned from (in this case, the GForge repository). If this was not the case, you can add an alias for the GForge repository URI with the “git remote” command:

~/gitproject1$ git remote add -m master gforge ssh://<username>@<gforge_server>/gitroot/gitproject1
~/gitproject1$ git remote -v
origin ssh://<username>@<gforge_server>/gitroot/gitproject1

In this case, “gforge” will be an alias for the remote GForge repository, and you’ll just have to specify “gforge” as the URI when pushing the changes:

~/gitproject1$ git push gforge master

Now, when you browse the git repository in the GForge project, you will be able to see the commits you’ve just pushed to the server.

Integration with GForge trackers

You may have noticed in the previous example that when we committed, the commit log included the special marker “[#16]”. GForge parses the commit logs and, whenever it finds such a string, it associates the commit with the specified item number, in this case 16.

If you take a look at the tracker item #16 (you can use the quick jump feature in GForge’s top navigation bar), you’ll see the commit information in the “Commits” tab inside the item:

You can click on the “Diff To” link and it will show you a page with the changes made to the file in that commit:

Setting up Access Control Lists

The Access Control Lists (ACLs) provide fine-grained permissions control in the source code repository, allowing you to specify read and write access for the different users in your project to different directories in the repository.

To set up an ACL for a directory, go to the Project’s git admin section by clicking on the “Git” section and then “Admin”. This will enable a new menu item “Edit repository ACLs”. Clicking on this item will take you to the ACL administration page.

To set up a new ACL, click on the “Add new ACL” button and specify the path for the ACL (it must be an absolute path in your source code tree). Once you define the path, you can edit the permissions for the path by clicking on the “Edit permissions” link for the ACL path. This will take you to a page where you can define the individual read/write permissions for each user in the project:

Whenever a user tries to push changes in a directory with read-only access, he will get an error message:

gitproject1$ git commit -a -m "Fixing stuff"
Created commit 1d32486: Fixing stuff
3 files changed, 3 insertions(+), 0 deletions(-)
gitproject1$ git push
<username>@<gforge_server>'s password: <your GForge password>
Counting objects: 13, done.
Compressing objects: 100% (7/7), done.
Writing objects: 100% (7/7), 690 bytes, done.
Total 7 (delta 4), reused 0 (delta 0)
User <username> doesn't have write access to /lib in the repository
error: hooks/pre-receive exited with error code 127
To ssh://<username>@<gforge_server>/gitroot/gitproject1
! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'ssh://<username>@<gforge_server>/gitroot/gitproject1'

In this example, the user has write access to the repository but the access to the /lib directory is read-only, as shown on the previous screenshot.

Setting up commit filters

The commit filters are custom scripts that run when a push is done against the repository in order to do code checks and any other action you want.

To define a commit filter, you need to be a site-wide administrator. On the Admin section of GForge, click on the SCM commit filters link under the Plugins tab. This will take you to a page where you can define the filters.

To add a new filter, click on the Add new SCM commit filter button. This will ask you for three things:

  1. Filter for: Whether this filter will be available for CVS, SVN or git repositories
  2. Description: A short description on what this filter does (e.g. “Check for Windows-style line breaks”)
  3. Command: The command to execute when a push is done against the server. This needs to include the full path to the script in the server and any additional parameters.
    There are three special parameters that are defined by GForge which you can add in the command line:

    1. $PROJECT: The UNIX name of the project (e.g. gitproject)
    2. $PATH: The full path to the repository, usually /gitroot/<project_name>.
    3. $FILE: (Not used in git)

Once the script is set up, you need to enable it in your project. Go to the project’s git admin page and click on the Edit commit filters link. In there, you can use the checkboxes to select which filters will be applied in your repository.