Code search in Gforge AS 5.7

One of the new features of Gforge AS 5.7 is the support for source code indexing/search using doxygen and tsearch2 in both Subversion and CVS repositories. In this article, we’ll go trough a complete example to show the highlights of this feature.


In order to use this feature, you’ll need doxygen. You can get it from your distribution repositories or compile it from source.

After installing doxygen, you may need to edit your /etc/gforge/gforge.conf file. Look for the $config['doxygenPath'] configuration variable, and change it as need.ed Then, recreate the configuration cache.

#cd /opt/gforge5 && php bin/create_config_cache.php

Setting the path for code indexing

If you want to enable the code indexing in a given project, you’ll need to set one or more source code paths. In this example we will use subversion, but you can accomplish the same with CVS.

Navigate to SVN->Admin->Code search paths, you will see a screen like this

Browse search paths

In this example we will use the source code from pgadmin, a graphical client for PostgreSql.
We are interested in the code under /trunk/pgadmin path, so let’s proceed to add the path.

Index path added

In this case we are using C++, but doxygen can handle several  other languages (C, Java, Objective-C, Python,PHP, etc), check the doxygen documentation for a complete list of supported languages.

Now that we have configured the path a cronjob executed on a daily basis will index all the code found under the specified path. However, you can run the indexing script manually:

#cd /opt/gforge5
#php bin/gforge cronjobs scm_index

Some searching examples

Now that we have all the source code indexed, we can search for classes, methods and properties in the last revision.

Let’s search for the class PgsRecord

First search

The search will return a long list of methods and properties for this class and the class itself, this is just a small sample

Search results

The fields returned are self-explanatory. Comments will be shown for a given item if there is any.

You can search for a specific method or property inside a given class by using :: or . operators (ie: PgsRecordset::insert and PgsRecordset.index are equivalents)

Second search example

In our final example, we will use the tsearch2 operators. Suppose you want to look for setters in PgsRecord or PgsParser classes, we can use the tsearch2 OR operator (|)

Third search example

git support in GForge

GForge provides full support for the git SCM (Source Code Management) tool, in addition to SVN and CVS.

Unlike SVN and CVS, git is a distributed tool where each user has a copy of the full repository in his or her machine. The basic workflow of git involves committing changes against the local repository and sharing the changes via pull and push operations.

Since commits in the local repositories cannot be “seen” by the GForge server, GForge sets up a global, centralized (“bare”) repository where the changes can be pushed to.

Setting up your project for git

To enable git support for your project, you need to be logged in as a project administrator. Get into the “Admin” section of the project, and select “GIT” as the SCM tool. Click the “Submit button”.

This will create a bare git repository in the GForge server. This may take up to 15 minutes, you can check if the repository was created by clicking on the “Git” section in the project. If you get the message “The SCM repository for this project hasn’t been created yet”, you need to wait a couple of minutes.

Once the repository is created, you will see the empty repository in the Git section:

Adding the initial data

At this point, the git repository is empty and is not very useful, so some data must be imported first. You will have to push an existing local git repository into the GForge repository.

For example, let’s assume you have an existing repository with source code in HOME/myrepo:

$ ls -a ~/myrepo
. .. bin conf cronjobs .git languages lib README.txt wwwlib

This is a working directory with the source code and the git repository is stored under the .git subdirectory.

Now, if you want to import this repository into the GForge project, you need to use the git push command, but first, you need to find out what’s the repository location (i.e. what’s the full URI for the repository in the GForge server). In the project, click on the Git section and then click on the Access info page. This wil give you the URI for the repository:

Note that the URI that’s used to clone the repository is the same you’ll use to push the changes.

So, with this information in hand, you can now push your repository to the GForge server:

~/myrepo$ git push ssh://<username>@<gforge_server>/gitroot/gitproject1 master
<username>@<gforge5.vm>'s password: <your GForge password here>
Counting objects: 113, done.
Compressing objects: 100% (104/104), done.
Writing objects: 100% (113/113), 86.39 KiB, done.
Total 113 (delta 51), reused 0 (delta 0)
To ssh://<username>@<gforge_server>/gitroot/gitproject1
* [new branch] master -> master

Note that this will only push the master branch to the repository.

The changes you’ve pushed will now be visible when you browse the repository:

Working with the git repository

Once the repository has some code, users can start working with it. For this, they must clone the repository from the GForge server. This is done with the git clone command as seen on the “Access info” page:

$ git clone ssh://<username>@<gforge_server>/gitroot/gitproject1
Initialized empty Git repository in /home/marcelo/gitproject1/.git/
<username>@<gforge_server>'s password: <your GForge password here>
remote: Counting objects: 113, done.
remote: Compressing objects: 100% (53/53), done.
remote: Total 113 (delta 51), reused 113 (delta 51)
Receiving objects: 100% (113/113), 85.51 KiB, done.
Resolving deltas: 100% (51/51), done.

This will create a directory with the project’s name (in this case gitproject1) with a local copy of the full repository.

The basic workflow with git involves 4 steps:

  1. Pulling changes from the GForge repository and updating the code
  2. Changing the source code
  3. Committing changes against the local repository
  4. Pushing changes to the GForge repository

For example, suppose someone has changed a files in their local repository and pushed them to the GForge server. The following describes a typical git session:

~/gitproject1$ git pull
remote: Counting objects: 7, done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 4 (delta 3), reused 0 (delta 0)
Unpacking objects: 100% (4/4), done.
From ssh://<username>@<gforge_server>/gitroot/gitproject1
70aa173..9450e42 master -> origin/master
Updating 70aa173..9450e42
Fast forward
wwwlib/index.php | 1 +
1 files changed, 1 insertions(+), 0 deletions(-)
~/gitproject1$ vi lib/ScmgitPlugin.php
(... edit, edit, edit...)
~/gitproject1$ git commit -a -m "Fixed bug [#16]"
Created commit d1a4acb: Fixed bug [#16]
1 files changed, 1 insertions(+), 0 deletions(-)
~/gitproject1$ vi cronjobs/create_git.php
(... edit, edit, edit...)
~/gitproject1$ git commit -a -m "Fixed another bug [#17]"
Created commit b8b281d: Fixed another bug [#17]
1 files changed, 4 insertions(+), 2 deletions(-)
~/gitproject1$ git push origin master
Counting objects: 13, done.
Compressing objects: 100% (8/8), done.
Writing objects: 100% (8/8), 777 bytes, done.
Total 8 (delta 5), reused 0 (delta 0)
To ssh://<username>@<gforge_server>/gitroot/gitproject1
9450e42..b8b281d master -> master

Note that when pushing, you usually don’t have to specify the full URI of the repository since it assigns the alias “origin” to the repository you cloned from (in this case, the GForge repository). If this was not the case, you can add an alias for the GForge repository URI with the “git remote” command:

~/gitproject1$ git remote add -m master gforge ssh://<username>@<gforge_server>/gitroot/gitproject1
~/gitproject1$ git remote -v
origin ssh://<username>@<gforge_server>/gitroot/gitproject1

In this case, “gforge” will be an alias for the remote GForge repository, and you’ll just have to specify “gforge” as the URI when pushing the changes:

~/gitproject1$ git push gforge master

Now, when you browse the git repository in the GForge project, you will be able to see the commits you’ve just pushed to the server.

Integration with GForge trackers

You may have noticed in the previous example that when we committed, the commit log included the special marker “[#16]”. GForge parses the commit logs and, whenever it finds such a string, it associates the commit with the specified item number, in this case 16.

If you take a look at the tracker item #16 (you can use the quick jump feature in GForge’s top navigation bar), you’ll see the commit information in the “Commits” tab inside the item:

You can click on the “Diff To” link and it will show you a page with the changes made to the file in that commit:

Setting up Access Control Lists

The Access Control Lists (ACLs) provide fine-grained permissions control in the source code repository, allowing you to specify read and write access for the different users in your project to different directories in the repository.

To set up an ACL for a directory, go to the Project’s git admin section by clicking on the “Git” section and then “Admin”. This will enable a new menu item “Edit repository ACLs”. Clicking on this item will take you to the ACL administration page.

To set up a new ACL, click on the “Add new ACL” button and specify the path for the ACL (it must be an absolute path in your source code tree). Once you define the path, you can edit the permissions for the path by clicking on the “Edit permissions” link for the ACL path. This will take you to a page where you can define the individual read/write permissions for each user in the project:

Whenever a user tries to push changes in a directory with read-only access, he will get an error message:

gitproject1$ git commit -a -m "Fixing stuff"
Created commit 1d32486: Fixing stuff
3 files changed, 3 insertions(+), 0 deletions(-)
gitproject1$ git push
<username>@<gforge_server>'s password: <your GForge password>
Counting objects: 13, done.
Compressing objects: 100% (7/7), done.
Writing objects: 100% (7/7), 690 bytes, done.
Total 7 (delta 4), reused 0 (delta 0)
User <username> doesn't have write access to /lib in the repository
error: hooks/pre-receive exited with error code 127
To ssh://<username>@<gforge_server>/gitroot/gitproject1
! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'ssh://<username>@<gforge_server>/gitroot/gitproject1'

In this example, the user has write access to the repository but the access to the /lib directory is read-only, as shown on the previous screenshot.

Setting up commit filters

The commit filters are custom scripts that run when a push is done against the repository in order to do code checks and any other action you want.

To define a commit filter, you need to be a site-wide administrator. On the Admin section of GForge, click on the SCM commit filters link under the Plugins tab. This will take you to a page where you can define the filters.

To add a new filter, click on the Add new SCM commit filter button. This will ask you for three things:

  1. Filter for: Whether this filter will be available for CVS, SVN or git repositories
  2. Description: A short description on what this filter does (e.g. “Check for Windows-style line breaks”)
  3. Command: The command to execute when a push is done against the server. This needs to include the full path to the script in the server and any additional parameters.
    There are three special parameters that are defined by GForge which you can add in the command line:

    1. $PROJECT: The UNIX name of the project (e.g. gitproject)
    2. $PATH: The full path to the repository, usually /gitroot/<project_name>.
    3. $FILE: (Not used in git)

Once the script is set up, you need to enable it in your project. Go to the project’s git admin page and click on the Edit commit filters link. In there, you can use the checkboxes to select which filters will be applied in your repository.