Source code management

From FreeCAD Documentation
(Redirected from GitHub)
Jump to: navigation, search
Other languages:
Deutsch • ‎English • ‎español • ‎français • ‎italiano

The main source code management tool for the FreeCAD project is Git, which can be easily installed in most operating systems from a package manager or directly from Git's website. You are advised to become familiar with Git before working with the FreeCAD source code directly. Visit the Git documentation page for the reference manual, as well as the Pro Git book to learn to use the system in a general way. The present document focuses on the use of Git for FreeCAD development. Compiling FreeCAD is described in Compiling.

While Git is primarily a terminal application, there are many graphical clients for it which facilitate working with branches, applying patches, and submitting pull requests to a master branch. Examples include gitk (the first graphical interface developed), gitg (Gnome), qgit (Qt), tig (Ncurses), git-cola, and GitKraken (proprietary). Please see Developing FreeCAD with GitKraken for a cursory introduction to this tool.

Source code access

Everybody can access and get a copy of the FreeCAD source code, but only the FreeCAD project managers have write access to it. You can get a copy of the code, study it and modify it as you wish, but if you want your changes to be included in the official source code, you need to perform a "pull request" against the master repository so that your modifications can be reviewed by the managers. This style of development is known as the Dictator and lieutenants workflow, as the core developers (dictators) and trusted developers (lieutenants) filter the code that is submitted by independent developers and users.

If your source code changes are significant, you are advised to explain them in the pull request section of the FreeCAD forum.

FreeCAD git workflow.svg

Generic workflow to develop code for FreeCAD; everybody can get the code from the main repository, but the main developers have the exclusive right to review and merge submissions by other developers.


Official GitHub repository

The FreeCAD source code is hosted in Github, https://github.com/FreeCAD/FreeCAD

In order to contribute code, you need to have a GitHub account.

Setting your Git username

Developers should commit code to their personal repository using their GitHub username. If that is not already set globally, you can set it locally for the current Git repository like this:

git config user.name "YOUR_NAME"
git config user.email GITHUB_USERNAME@users.noreply.github.com

Where "YOUR_NAME" represents your full name or nickname, used to identify the author of a particular commit, and GITHUB_USERNAME indicates the name of your account on GitHub.

Remote repositories

Please read What is the difference between origin and upstream on GitHub? (Stackoverflow) to help you understand the difference between origin and upstream in the context of Git. This section explains how to set the correct repositories for development. Essentially:

  • origin is your personal fork of the official FreeCAD repository, that is, https://github.com/GITHUB_USERNAME/FreeCAD
  • upstream is the official FreeCAD repository, that is, https://github.com/FreeCAD/FreeCAD

This distinction is important, as you should write code in your own copy of the repository first, before pushing those changes to the official repository.

Based on the above, there are two ways to setup your Git development environment:

  • 1st Method: fork on GitHub and clone your fork locally
  • 2nd Method: clone FreeCAD directly to your local machine, and adjust the remote servers

We recommend the 1st method because it's one step faster.

1st Method: Fork on GitHub and clone your fork locally

First you will fork the FreeCAD repository in GitHub, then clone this personal fork to your computer, and finally set the upstream repository.

  • Log in to your GitHub account.
  • Go to the official FreeCAD repository: https://github.com/FreeCAD/FreeCAD
  • In the top right of the page press the "Fork" button. This will create a personal copy of the FreeCAD repository under your GitHub username: https://github.com/GITHUB_USERNAME/FreeCAD
  • On your machine, clone your newly created FreeCAD fork. It will be created inside a directory freecad-source.
git clone https://github.com/GITHUB_USERNAME/FreeCAD.git freecad-source
  • Once the download is complete, enter the new source directory and set the upstream repository.
cd  freecad-source
git remote add upstream https://github.com/FreeCAD/FreeCAD.git
  • Confirm your remote repositories with git remote -v; the output should be similar to this
origin	https://github.com/GITHUB_USERNAME/FreeCAD.git (fetch)
origin	https://github.com/GITHUB_USERNAME/FreeCAD.git (push)
upstream	https://github.com/FreeCAD/FreeCAD.git (fetch)
upstream	https://github.com/FreeCAD/FreeCAD.git (push)
  • Now development can begin.

2nd Method: Clone FreeCAD directly to your local machine

First you will fork the FreeCAD repository in GitHub, however, you will clone the original FreeCAD repository to your local machine, and then alter your remotes via the terminal.

  • Log in to your GitHub account.
  • Go to the official FreeCAD repository: https://github.com/FreeCAD/FreeCAD
  • In the top right of the page press the "Fork" button. This will create a personal copy of the FreeCAD repository under your GitHub username: https://github.com/GITHUB_USERNAME/FreeCAD
  • Clone the original FreeCAD repository. It will be created inside a directory freecad-source.
git clone https://github.com/FreeCAD/FreeCAD.git freecad-source
  • Once the download is complete, enter the new source directory and set the origin repository.
cd freecad-source
git remote add origin https://github.com/GITHUB_USERNAME/FreeCAD.git
  • Then set up the upstream repository.
git remote add upstream https://github.com/FreeCAD/FreeCAD.git
  • Confirm your remote repositories with git remote -v; the output should be similar to this
origin	https://github.com/GITHUB_USERNAME/FreeCAD.git (fetch)
origin	https://github.com/GITHUB_USERNAME/FreeCAD.git (push)
upstream	https://github.com/FreeCAD/FreeCAD.git (fetch)
upstream	https://github.com/FreeCAD/FreeCAD.git (push)
  • Now development can begin.

If for some reason the remote repositories exist but point to the wrong address, you can remedy the situation by renaming the remote repository's name. For example, origin should point to your personal fork; if it is pointing to the original FreeCAD repository, change the name of this remote to upstream, and manually add the origin repository.

git remote rename origin upstream
git remote add origin https://github.com/GITHUB_USERNAME/FreeCAD.git
git remote -v

You can also show more information with the show keyword.

git remote show origin
git remote show upstream

Git development process

Never develop on the master branch. Instead, create a branch for development, and then merge this branch to the master branch. Please read Git Branching, Basic Branching and Merging, and GitHub - Contributing to a project to learn more.


FreeCAD git branches workflow.svg

Generic workflow to develop code for FreeCAD using git; the main repository is forked online and cloned to an offline computer (0); new branches (1) are used to commit local changes and additions to the code (2); the branches are rebased to the latest online code (3), and then are pushed to the remote repository (4); then a pull request is created in order to merge the code into the main repository (5). Then the personal clone is updated with the new master code (a); this updated master is also pushed to the remote repository (b) in order to have the same code both online and offline.


Branching

Instead of working on the master version of the code, best practices with Git recommend creating a new branch whenever you want to work on a new feature. Branches are inexpensive, they don't copy the entire source tree, but merely create a point in time on top of which you will write code; thus branches help keep work in progress separate from the main code.

Using a new branch is done in two steps, first your create the branch, and then you switch to it:

git branch myNewBranch
git checkout myNewBranch

Alternatively, perform both steps with a single instruction:

git checkout -b myNewBranch

Now you can change branches with checkout whenever you need to work on them. To see the branches in your project and the current branch, use the branch operation alone, or add -v or -vv for more information:

git branch
git branch -vv

After you've made changes and committed those changes use the log operation with the following options to visualize the branches

git log --oneline --decorate --graph --all

Committing

Once you are inside a new branch, edit the source files that you want with a text editor. To see which files were modified use the status and diff operations; when you are satisfied with the modifications, save the changes with the commit operation:

git status
git diff
git commit -a

Unlike SVN, you need to specifically tell which files to commit; use the -a option to save changes in all files that were altered. Your text editor, for example, nano or vim, will open to allow you to write a commit message.

Alternatively add the message in the commit itself

git commit -a -m "Fix the bug in the clone function."

Writing good commit messages

You should try to work in small steps, that is, commit often, after a small addition in your code. If you cannot summarize your changes in one sentence, then it has probably been too long since you made a commit.

For big changes, it is important that you have helpful and useful descriptions of your work. FreeCAD has adopted a format mentioned in the Pro Git book, which consists of a short message, and then a larger descriptive paragraph.

Short (50 chars or less) summary of changes
 
 More detailed explanatory text, if necessary.  Wrap it to about 72
 characters or so.  In some contexts, the first line is treated as the
 subject of an email and the rest of the text as the body.  The blank
 line separating the summary from the body is critical (unless you omit
 the body entirely); tools like rebase can get confused if you run the
 two together.
 
 Further paragraphs come after blank lines. 
 
  - Bullet points are okay, too
 
  - Typically a hyphen or asterisk is used for the bullet, preceded by a
    single space, with blank lines in between, but conventions vary here

If you are doing a lot of related work in a branch, you should make many small commits (see a forum post). When you want to merge those changes into the master branch, you should issue

git log master..myNewBranch

to see the individual commit messages. Then you can write a high quality message when performing a merge.

When you merge to master use the --squash option and commit with your quality commit message. This will allow you to be very liberal with your commits and help to provide a good level of detail in commit messages without so many distinct descriptions.

Squashing commits

Squashing refers to the process of combining various consecutive commits into one. This may be desirable if you made many small commits that you want to present as a single commit, for example, when changing a single variable, correcting spelling mistakes, and adjusting the spacing of the code. You should squash only small commits to a single file; big changes to the code across multiple files should contain the full commit history.

With git log --oneline you can see many commits in sequence, with the newest commit on top. In this example, starting from "feature A" many commits are made to implement "feature B"; we would like to squash all commits belonging to "feature B" into one.

871adb OK, feature B is fully implemented
1c3317 Whoops, it is not ready yet...
87871a I'm almost ready!
643d0e Code cleanup
af2581 Fix this and that
4e9baa Good implementation
d94e78 Prepare the module for feature B
6394da Feature A

Use the rebase operation with the --interactive option to select various commits and squash them. Use the hash of the commit just before the first one that you want to squash, in this case the one corresponding to "feature A".

git rebase --interactive 6394da

The command line editor, like nano or vim, will open to show you the commits again, now with the older commit on top. Before each commit, the word pick will be shown. Delete the word pick, and write the word squash or just the letter s instead, with the exception of the first entry; this commit is the oldest one, so all future commits will be squashed into it.

pick d94e78 Prepare the module for feature B
s 4e9baa Good implementation
s af2581 Fix this and that
s 643d0e Code cleanup
s 87871a I'm almost ready!
s 1c3317 Whoops, it is not ready yet...
s 871adb OK, feature B is fully implemented

Save the file and close the editor.

The editor will open up again. Now you can add a longer message that describes all changes as if they were a single commit. Save the file and close the editor once more. This will finish combining those commits into one, with the new commit message that you wrote.

You can use git log --oneline again to observe the new commit history. In this case only a single commit for "feature B" will appear, on top of the unmodified commit for "feature A".

c83d67 OK, feature B is fully implemented now, with proper module setup, and clean code.
6394da Feature A

Pushing your work to your GitHub repository

The local branches in your computer aren't automatically synchronized with the remote servers that you have specified as origin or upstream (see Remote repositories); you have to explicitly push the branches to the remote servers, for which you must have write access. Once you do this, the branches become public, and available for review by other developers.

For FreeCAD, you should push your local branch to the origin remote repository, that is, https://github.com/GITHUB_USERNAME/FreeCAD. You need to enter your username and password every time you push, unless you have set up Credential caching. Please read Pushing commits to a remote repository for more information.

git push origin myNewBranch

The regular developer doesn't have write access to the upstream repository https://github.com/FreeCAD/FreeCAD, therefore, you shouldn't push code to this remote server.

Rebasing from upstream

While you work on your own branch, the official FreeCAD code keeps "moving forward" with commits from other developers, and thus starts diverging from the code that you have in your personal fork.

      .-----A origin/myNewBranch
     / 
-----o-----------Z FreeCAD upstream/master

Therefore, when you are ready to merge your branch to the main FreeCAD repository, you must "rebase" your own copy of the repository, so that it is as close as possible to the official repository. See Git Branching - Rebasing for more information.

git checkout myNewBranch
git pull --rebase upstream master

This will download the code from the master branch of the upstream repository (the official FreeCAD source), and will merge it with your current branch (myNewBranch), so that your changes will appear on top of the latest official code. If nobody modified the same files that you did, then the merge will succeed without problems. If some files were changed at the same time by different people, there may be a conflict that needs to be resolved.

                  .-----A' origin/myNewBranch
                 /
-----o-----------Z FreeCAD upstream/master

To summarize, you need to be in the appropriate branch, rebase the upstream code, and then proceed with the push.

git checkout myNewBranch
git pull --rebase upstream master
git push origin myNewBranch

The pull operation is equivalent to a fetch followed by a merge. When the --rebase option is used, instead of doing a simple merge, it runs the rebase operation.

git pull upstream

git fetch upstream
git merge FETCH_HEAD
git pull --rebase upstream master

git fetch upstream
git rebase master

Merging the branch (pull request)

Once you have committed your changes locally, rebased your branch from the upstream repository, and pushed your branch online, you can initiate a "pull request". A pull request tells the administrators of the official FreeCAD repository that you want to merge the new code in your branch with the official code.

As soon as you push the code to your origin repository https://github.com/GITHUB_USERNAME/FreeCAD, GitHub will give you the option of comparing and creating a pull request against the upstream repository. By pressing Compare & pull request you will open an interface that will allow you to pick which repository is the "base", target of the merge, and which is the "head", your additional code. A quick check will be done by the system telling you if there are no conflicts with the files that you modified; if you worked on files that nobody has touched, your branch will be able to merge cleanly. In addition, it will show you a text editor so you can write a message documenting your changes; it will also display the number of commits in your branch, the number of files that were modified, and a view showing you the differences between the "base" and the "head" so that everybody can immediately see your intended modifications.

base repository: FreeCAD/FreeCAD    base: master  <----  head repository: GITHUB_USERNAME/FreeCAD    compare: myNewBranch

Able to merge. These branches can be automatically merged.

Click Create pull request to proceed. A message will appear indicating that some checks need to be done on the code. This is a system that compiles FreeCAD automatically and runs the unit tests. If the tests pass, the pull request will have a better chance of being merged into the main code, otherwise a report will be made indicating the errors encountered. See FreeCAD pull requests.

Some checks haven’t completed yet

* continuous-integration/travis-ci/pr Pending — The Travis CI build is in progress  |Required|

If the tests succeed, you will see a message such as the following

All checks have passed

* continuous-integration/travis-ci/pr — The Travis CI build passed  |Required|

This branch has no conflicts with the base branch
Only those with write access to this repository can merge pull requests.

Now you must wait for the administrators to merge your branch; you will be notified when this happens.

Pull request successfully merged and closed

You’re all set — the GITHUB_USERNAME:myNewBranch branch can be safely deleted.
If you wish, you can also delete your fork of FreeCAD/FreeCAD.

If you wish, you may delete the branch that was just merged, or even your entire FreeCAD fork, as your own code is already included at the end of the master branch.

-----o-----------Z----A' FreeCAD upstream/master

Note: you may continue working (git commit -a) on the same branch while you wait for merge approval; if you git push again, a second merge commit will be queued in the same pull request, and another automated test will be done. That is, while your merges aren't yet approved by the administrators, you may keep pushing changes to your origin repository, and this will queue those commits in the same pull request to the upstream repository. Using a single pull request to queue many individual commits is often desirable for small changes. For big additions to the source code, you should create another branch, develop your features there, and then submit a separate pull request for this branch.

The pull request interface can be used whenever you want to submit code from your own repositories to another repository in GitHub. You can use it to merge code in the opposite direction as well, from other people's branches to your own, or even between your own branches. In the last case, since you own the branches, the merges can be approved by yourself immediately.

base repository: SomeProject/Some_Software  base: master       <----  head repository: GITHUB_USERNAME/Some_Software  compare: add_new_functions
base repository: GITHUB_USERNAME/FreeCAD    base: myNewBranch  <----  head repository: FreeCAD/FreeCAD                compare: master
base repository: GITHUB_USERNAME/FreeCAD    base: myNewBranch  <----  head repository: GITHUB_USERNAME/FreeCAD        compare: fix-many-bugs-branch

Keeping the GitHub repository up to date

Once you've forked FreeCAD, your personal repository exists independently from the original. When the original repository has new commits, GitHub will inform you that your personal repository is behind in number of commits:

This branch is 5 commits behind FreeCAD:master.

In similar way, if you created a development branch with new code, GitHub will inform you that this branch is ahead in number of commits; that is, this branch has changes that haven't been merged into the official FreeCAD repository:

This branch is 3 commits ahead of FreeCAD:master.

While developing, both cases are possible, as your own branch may lack commits made by other developers, but include new commits by you:

This branch is 2 commits ahead, 14 commits behind FreeCAD:master.

When developing code it is recommended that you rebase the branch in which you are currently working, as that will put your branch always ahead of the FreeCAD master code.

As for your original master branch, it will never be automatically updated by GitHub; this is something that you must do yourself. Switch to the master branch, then pull from upstream (which performs a fetch and merge), and then push this updated master branch to your remote origin repository.

git checkout master
git pull upstream master
git push origin master

After this is done, GitHub will let you know that your are synchronized with the upstream repository.

This branch is even with FreeCAD:master.

Now that your master is up to date, you may decide to switch to it, and delete the other branch that you used previously to develop a feature.

git checkout master
git branch -d myNewBranch

To delete the branch in the origin remote repository, you can use the push action. Normally, you push a local branch; this creates a remote branch with the same name as your local branch.

git push origin myNewBranch

However, if you use the notation local_name:remote_name, the local branch is created in the remote repository under a different name:

git push origin myNewBranch:someRemoteBranch

Therefore, you can delete the remote branch by pushing an empty local branch:

git push origin :myNewBranch
git push origin :someRemoteBranch

Now that you only have an up-to-date master, you can create a new branch, and repeat the steps of changing files, committing, pushing, submitting a pull request, merging, and updating.

git checkout master
git checkout -b anotherBranch

If you don't want to delete your already custom branch, you may force updating it to be equal to the updated master; then you can do whatever you want with it, including adding more commits and pushing it to the remote origin repository.

git checkout myNewBranch
git reset --hard master
git push -f origin myNewBranch

Hard resetting a branch like this is usually not needed. In most cases, you want to follow the sequence of creating a new branch, committing changes, pushing those changes, merging the branch, and then deleting the branch.

Advanced Git operations

Resolving merge conflicts

Merging local branches with git merge will occasionally present conflicts, as the files may have been changed at the same time in the same place. If this happens you should fix the files, and re-merge.

Inspect the changes of a single file through various commits:

git log -p -- myFile.cpp

Creating patches from Git

Although Git allows you to merge different branches of code with git merge (in your computer) or a pull request (remote repository), there are times when it may be desirable to create a traditional "patch", which can be sent as an attachment through email. The following workflow explains how to do this.

  • You should be developing your new code in a secondary branch of your repository, and not in the master branch. So the first step is to make sure you are in the correct branch.
git branch -v
git checkout myBranch
  • Now use git format-patch against the master branch, and use the --stdout option to redirect the result to standard output; then redirect the standard output to a file, which for convenience is created above the source code directory.
git format-patch master --stdout > ../myCode.patch
  • Another method is
git format-patch HEAD^
git format-patch HEAD~1

The number of circumflex carets ^ or the number 1 indicate the number of commits that should be considered, that is, ^^^ or ~3 will create three patches for three commits.

git format-patch HEAD^

This will create a patch or series of patches with the following naming convention

XXXX-commit-message.patch

where XXXX is a number from 0000 to 9999, and the commit message forms the majority of the file name, for example,

0001-fix-ViewProjMatrix-getProjectionMatrix.patch

Applying patches via git

Git can merge patches or diffs. To know more about this process read Applying patches with Git.

If you already have the patch file in your system, just apply it.

git apply myCode.patch

You can use curl to download a patch from a website, and then apply it through git.

curl -O https://some.website.org/code/myCode.patch
git apply myCode.patch

Add .diff or .patch at the end of the URL of a GitHub commit, pull request, or compare view so that the website shows you the plain text view of that page.

You can point curl to a particular commit patch in the repository, and pipe it directly to git to apply the patch.

curl https://github.com/FreeCAD/FreeCAD/commit/c476589652a0f67b544735740e20ff702e8d0621.patch | git apply -

Reversing a patch in git

When you apply a patch you modify some files. However, these modifications aren't permanent until you commit the changes. Therefore, if you want to revert a patch use the following instructions.

This will revert the changes applied, if you still have access to the original patch file.

git apply -R myCode.patch

Alternatively, this will remove non-committed changes to the branch.

git checkout -f

Stashing git commits

Say that you're working on a branch and you find yourself making some modifications to the source that are out of the scope of your current branch; in other words, those changes would be better in another branch instead of the current one. The git stash command can be used to temporarily store those uncommitted local changes.

git stash
git stash push

If in the future you want to use those commits, you can pop the commits out of the stash, and into your working branch.

git stash pop

Or if you decide that you don't like those saved commits anymore, you may drop the commits from the stash entirely.

git stash drop

You can list multiple stash commits with

git stash list

To learn more, read Useful tricks you might not know about Git stash.

Check out GitHub requests locally

Checkout GitHub pull requests locally

FreeCAD revision number

In contrast to subversion, which uses a consecutive number for its revisions, Git produces SHA-1 hash values with every commit. A hash value is a long alphanumeric string that looks like this

9b3ffef570596e184006287434fba54a4b03ccc3

Latest revision number

To find the latest revision number of a particular branch use the rev-list operation with the --count option. Give the name of the branch, remote repository, tag, or a special pointer like HEAD, to indicate the last commit in that particular object.

git rev-list --count master
git rev-list --count HEAD
git rev-list --count origin

Or browse the repository on GitHub, and read the amount of commits reported in the particular branch.

Revision number of a specific commit hash

Since the hash is an alphanumeric string it is not very useful to decide if a certain commit is older or newer than another hash. To find the revision number of a particular hash, again use the rev-list operation; the input can be the full hash, or a partial hash that is unique, usually the first 7 digits are enough.

git rev-list --count ab1520b872821414c6ce4a15fb85d471ac2a2b03
git rev-list --count 9948ee4

Revision hash of a specific commit number

If we have the commit number, say, 15000, and we want to find the corresponding hash, we need to calculate the number of commits since this point until the last commit (HEAD). First, get the latest commit number.

git rev-list --count HEAD
17465

Then subtract the commit that we want.

17465 - 15000 = 2465

Then use the log operation to show all commits and hashes. The --skip option jumps the difference in commits that we calculated so that we go directly to the hash that we are looking for.

git log --skip=2465
commit 44c2f19e380e76b567d114a6360519d66f7a9e24

Since the log may show you two close commits, confirm it's the right commit number. If it's off by one, just pick the next commit in the sequence (before or after) and check again.

git rev-list --count 44c2f19e38
15000

Revision number in FreeCAD's interface

The version number that appears in Help → About FreeCAD → Copy to clipboard is defined in src/Build/Version.h, which is created at compile time when the cmake tool is run. Read Extract version number from git source for more information.

See the compile on Unix page for the full information on compiling FreeCAD.

Alternative repositories

Several collaborators of the FreeCAD project have their own Git repositories where they build up their work or where they experiment new ideas before they are ready to be included in the official source code. In certain cases you might want to clone your FreeCAD sources from these other repositories to test their code.

Use the git remote command to add these other repositories so that you can pull their changes.

git remote add Johns_repo https://github.com/John/FreeCAD.git
git remote add FreeCADhacker_repo https://github.com/FreeCADhacker123/FreeCAD.git

Head to the development section of the FreeCAD forum to discuss more about development of new features. See also Compiling for instructions on compiling FreeCAD.

Further reading