Tuesday, April 13, 2010

Set up git on Synology NAS

I just installed a NAS at home.
It's going to be my primary Data Silo so naturally I need to store my (backup-/published-) git repos there.

The Synology thing I bought comes with Linux/Busybox. Hacking around is surprisingly easy, they actually encourage you to do it.

Of course I could have gone the easy way of nfs-mounting my shares and then have used local paths to update the repos. But I wanted to be able to do this from the road using ssh. So I needed git to be Installed.

Here is what I needed to do to install git on the Box:
  1. Open ssh. Easy its in the GUI.
  2. Create users and shares
  3. Set up your User accounts to:

    • Have a proper Home Directory
    • Be able to log in using Public/Private keys
  4. Configure Additional Software Repositories
  5. Install git
  6. Configure sshd so that git+ssh works

1. Open ssh


Open the Web interface of the Box.
Select the "Management" View.
Go to "Network Services" -> "Terminal"
Select "Enable SSH service" and click "OK"
done

2. Create users and shares
Go to "Privileges" -> "User" and create a User
We need at leas one share to be able to transfer a File to the Box.

3.Set up your User accounts
Use ssh to log in as "root" on the box. Use the "Admin" password that you provided when
Initially setting up the NAS.

You will see the Busybox promt:
BusyBox v1.1.0 (2010.03.12-16:58+0000) Built-in shell (ash)
Enter 'help' for a list of built-in commands.

syn>

create home directories for your users:
mkdir -p /volume1/home/user1

Then edit /etc/passwd to enable the user for ssh use. We change he home dir set the login shell:
...
user1:x:1026:100::/volume1/home/user1:/bin/sh
...

By the way. /bin/sh is NOT Bash here. Its Busybox (ash).
Now you should be able to access the NAS through ssh with that user.

For convenience we want to be able to use Public/Private keys for login:

Edit /etc/ssh/sshd_config and change the Lines regarding Public Key Authentication:
...
PubkeyAuthentication yes
AuthorizedKeysFile      .ssh/authorized_keys
...


Restart sshd:
/usr/syno/etc/rc.d/S95sshd.sh restart

Oddly this sometimes only works half way. If your sshd does not come back, simply switch it back on in the Web interface.

Now copy your public key to a share and move it to the authorized_keys file:
mkdir -p /volume1/home/user1/.ssh
mv /volume1/someshare/id_dsa.pub /volume1/home/user1/.ssh/authorized_keys

Now you should be able to log in without a password.

4. Configure Additional Software Repositories

Pretty straight forward as described in the Synology wiki

Copy the boostrap file to a Share ( syno-mvkw-bootstrap_1.2-7_arm.xsh), then run it as root on the box:
sh /volume1/someshare/syno-mvkw-bootstrap_1.2-7_arm.xsh



5. Install git
Look for the git package:
ipkg list|grep git
dcraw - 1.376-2 - Decoding raw digital photos.
digitemp - 3.6.0-1 - Reads 1-Wire Temperature sensor (http://www.digitemp.com)
git - 1.6.6.2-1 - GIT is a directory tree content manager that can be used for distributed revision control.
git-lite - 1.6.6.2-1 - GIT is a directory tree content manager that can be used for distributed revision control.
git-manpages - 1.6.6.2-1 - manpages of git
git-svn - 1.6.6.2-1 - git as svn client
gitosis - git20090917-1 - Git repository hosting application.
gphoto2 - 2.4.1-1 - Command line digital camera software applications
libdlna - 0.2.3-2 - Reference DLNA (Digital Living Network Alliance) open-source implementation for Linux.
libgphoto2 - 2.4.1-1 - digital camera software libraries
sane-backends - 1.0.20+git20091022-1 - SANE is a universal scanner interface
squeezecenter - 7.3.3-1 - Streaming Audio Server for Logitech Squeezebox
stgit - 0.14.3-2 - StGit is a Python application providing similar functionality to Quilt (i.e. pushing/popping patches to/from a stack) on top of
tig - 0.15-1 - Tig is a git repository browser that additionally can act as a pager for output from various git commands.

Install it:
ipkg install git


Now we can create a repo for publishing. Log in as your user:
mkdir -p /volume1/git/test
cd /volume1/git/test
git init --bare
Initialized empty Git repository in /volume1/git/test/


6.Configure sshd so that git+ssh works
Now git works in regular login shells. When using git to clone the new repo this fails:
git clone ssh://192.168.178.29/volume1/git/test
Initialized empty Git repository in /data/t/.git/
sh: git-upload-pack: not found
fatal: The remote end hung up unexpectedly

This means that the "git-upload-pack" on the NAS was not found in the $PATH.
The ipkg Packages are installed to /opt/(s)bin which is not in sshds standard path.
Since sshd for BusyBox was configured not to parse /etc/profile or ~/.profile we have to fall back to providing a ~/.ssh/environment file with the correct path:
PATH=/opt/bin:/opt/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/syno/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/syno/bin:/usr/syno/sbin:/usr/local/bin:/usr/local/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/syno/bin:/usr/syno/sbin:/usr/local/bin:/usr/local/sbin

The Path can be copied from /root/.profile
To switch on user environments in sshd, configure /etc/ssh/sshd_config :
...
PermitUserEnvironment yes
...

Then restart sshd.

Have fun!

42 comments:

Chris said...

Very useful guide. Thanks a lot!

Drew Noakes said...

Thanks for this. Exactly what I need.

Drew Noakes said...

Finally got around to this earlier today. One point, make sure that the directories you create (/volume1/home/user1 and /volume1/home/user1/.ssh) and any contained files are all owned by 'user1' and not 'root' otherwise you won't be able to log in with your key. When using PuTTY I was seeing the message "Server refused our key". I used chown to get around this.

dugi said...

Hey,

Awesome guide! Been searching for this for a while now. I have stumble upon a little issue. After I installed git - which it did successfully.

Then I log out of root and back ind with "user" and tries to create the "/volume1/git/test" folder and I get a "Permission denied". And I can't see any "git" folder in my "volume1" either.

I'm not so experienced with the use of Busybox or anything like that, so please bear over with me :)

In advance, thank you and again, AWESOME job with this guide!

Regards,
Morten

Chris said...

Hi Morten,

can you see the /volume1/git folder with the root user?

If yes you might have to check permissions on /volume1 [/git/...]

ls -la / |grep volume

gives me the following on my box:
lrwxrwxrwx 1 root root 13 Aug 9 01:09 data -> /volume1/data
lrwxrwxrwx 1 root root 12 Aug 9 01:09 git -> /volume1/git
lrwxrwxrwx 1 root root 13 Aug 9 01:09 home -> /volume1/home
lrwxrwxrwx 1 root root 12 Aug 9 01:09 svn -> /volume1/svn
drwxr-xr-x 14 root root 4096 Jan 7 21:24 volume1
drwxr-xr-x 2 root root 4096 Jan 5 12:50 volumeUSB1
drwxrwxrwx 2 root root 4096 Jan 1 02:07 volumeUSB2
drwxrwxrwx 2 root root 4096 Jan 5 09:49 volumeUSB3


you need at least rwxr-xr-x on /volume1 for the user to be able to go into that directory. You might be able to configure that in the DSM Web interface. Look in the shares menu.
then check that all folders you need to access have write permissions.

One easy way of doming this is by setting the "Owner" of the directory to your user.

use something like chown -R user.users /volume1/git

fsckin said...

Brilliant. Thanks for this.

Simple typo: ipgk install git

Chris said...

corrected, thanks!

Christopher said...

I'm stuck at point 6.
I get the following error:

"error: cannot run /opt/bin/ssh: No such file or directory"

What path do I have to copy where? Do I have to create a new file and directory? Thank you very much!

Chris said...

do a

cat /root/.profile
look for the PATH= variable

create a new file in your user home directory ~/.ssh/environment and put that path variable there.

This is basic Linux stuff. If you don't know what all this means its probably a good idea to leave it be. In case something breaks you might not be able to fix it.

Christopher said...

Thank you for your quick reply, unfortunately this doesn't seem to work... I get the following path variable:

DiskStation> cat /root/.profile
#/etc/profile: system-wide .profile file for ash.
PATH="$PATH:/usr/bin:/bin:/usr/sbin:/sbin:/usr/syno/bin/bin:/sbin:/usr/bin:/usr/sbin:/usr/syno/bin:/usr/syno/sbin:/usr/local/bin:/usr/local/sbin"

I then created an environment file in ~/.ssh/

DiskStation> cat ~/.ssh/environment
PATH="$PATH:/usr/bin:/bin:/usr/sbin:/sbin:/usr/syno/bin/bin:/sbin:/usr/bin:/usr/sbin:/usr/syno/bin:/usr/syno/sbin:/usr/local/bin:/usr/local/sbin"

Do I have to add the /opt/bin:/opt/sbin:/ in them?
problem is that I checked and there really is no ssh in /opt/bin/ and I still get that error everytime I try to do a git clone:

error: cannot run /opt/bin/ssh: No such file or directory
fatal: unable to fork

It's true I'm not very experienced with linux, but I would really like to get this working...

So if you could point me in the right direction that would be great! Thank you loads!

Derek said...

Thanks for the guide. I'm still picking out a NAS box to be my primary data store in the house and Synology keeps coming up. What model do you have?

Chris said...

it's a 210j.
But I am in a move away from that:

3 Reasons: 1. I am in (Suse) Linux Consulting. So hacking comes with the job description. There are certain things that are odd with a pre made thing like the synology boxes while still these are the most hackable yet.

2. I bought the box with one (Advertised) feature in mind: rsync backup over wan(internet). Unfortunately Synology its still broken: It only works with rsyncd + open ports on the receiving end. That's not an option. I opened tickets and explained to them why NOBODY does this. They simply don't get it. The sad thing: Its only the GUI that is at fault. The rsync binary on the box is capable of that.

3rd. I am out of space. -> I need more spindles.

Still the syno Boxes are unbeatable when it comes to power consumption and Features/Eur.

cu
Chris

Derek said...

Can you go into more detail about the rsync issue? I was definitely planning to do backups via rsync over ssh. I'm Linux literate but don't know about the issue regarding rsync and open ports.

Also what odd things have you found? I'm really drawn to the DS211J as an ideal NAS box for me. 4TB I've calculated is enough for my foreseeable needs. Its a bit pricey, but as you said features/cost its as good as I've found.

Chris said...

The people at Synology simply don't get that what they call "rsync compatible Server" Is in fact a very rare thing: It needs to run the rsync deamon and has to be accessible through the rsync port (873). The normal thing today is to simply use rsync over ssh (even the rsync manual says this). This way you dont have to open additonal ports.
The oddest thing is: The "Enable transfer Encryption" Checkbox in the Syno Gui uses ssh but still requires you to configure rsyncd.

As to 4TB is enough: That will be only 2TB since redundancy is a must at the current HDD prices ;-)

P5ycH0 said...

I am going for the setup without authorized keys.
So, I installed git on my synology and try to run "git clone ssh://root@ip-address-of-server/volume1/git/test" on my client.
I copied the path of my root account into the .ssh/environment file of my root account. I've even set "PermitUserEnvironment yes" in the "/etc/ssh/sshd_config" file.
However, I'm still getting "ash: git-upload-pack: not found".

What more can I try?

P5ycH0 said...

Update:
I've fixed it by creating simlinks in /usr/bin:
As root:
cd /usr/bin
ln -s /opt/bin/git* .

It's a workaround, but it works.

So, the environment file & the PermitUserEnvironment didn't work for me.

Chris said...

I'm am getting the feeling that there are some significant differences in your box to mine.
It might have to do with DSM Version on my Production NAS I still run 2.3. I upgraded another one to 3.0 and will try to reproduce my steps.

de59920c-7e69-11e0-a1a0-000bcdcb2996 said...

i am lost i have no idea how to get the id_dsa.pub file and wheni create a user it says that user doesnt exist on ssh

Wayne said...

Very useful post, thanks!

@P5chHO - thanks for that workaround. I was getting "git receive-pack not found" when doing a git push ssh://username@synology/... even though I had added in the /opt/bin the ~/.ssh/environment file (for both the user and root account). I even ssh'ed in (as both username and root) and verified that ash and sh found git-receive-pack on their paths ... I was starting to dig into the mystery when I saw your 'workaround' so you've probably saved me a fair bit of head-banging!

Fouppy said...

Hi,

Great post, except the fact that i have a problem, the same than Christopher's.

I've managed to add the PATH variable to ~/.ssh/environment and still have the same error :

error: cannot run /opt/bin/ssh: No such file or directory
fatal: unable to fork

Any help would be much appreciated.

Thanks !

dugi said...

I run into the same problem, with the annoying:

error: cannot run /opt/bin/ssh: No such file or directory
fatal: unable to fork


Has anyone found a solution for this? I see that someone has found a fix - could maybe @Wayne, @P5chHO or anyone else C/P the fix for us, that aren't too experienced with symlinks etc? :)

It would be much appreciated!

Thanks :)

Stefan said...

Hi! great post! I would appreciate an update to match latest version of the synology :)

maurycy said...

Hello,

I added the PATH vars and also resorted to @P5chH0's tip however I still am receiving a git-receive-pack not found and/or git-update-pack not found. Any help or direction would be greatly appreciated.

compagnon said...

to restart ssh, use :
/usr/syno/sbin/synoservice --restart ssh

0x80 said...
This comment has been removed by the author.
0x80 said...

Probably a stupid question but where do you get the id_dsa.pub from? Do I need to generate one?

0x80 said...

Nevermind. Thanks for the great tutorial. It was a bit hard for me since I'm not very familiar with git/ssh/linux in general, but with a bit of patience and extra help from google I managed to figure it out after all.

竹板凳 said...

you are a life saver. thanks a bunch.

Brendan said...

You might want to turn off "StrictModes" in sshd_config, if public key authentication doesn't work for you. I found it necessary with my DS211j

Dave W said...

Great tutorial - many, many thanks for taking the time to write this up.

Arjan said...

I found this tutorial while figuring out what GIT is.
First: this key that you get through your share, is that the key created with the Windows GIT program?
Second, when you use the home directory service of Synology there is a homeS directory with the users in it. Do you mean this directory or can you use it, or does it have to be the home directory?

101385282250000896290 said...

First of all, thanks for a very detailed and useful post.

I think I found the solution to the issue with git PATH issues. If you configure the sshd_config as detailed above, it should work.

But if your home directory for your user in /etc/passwd is setup to something like /volume1/git_repo for instance, then you need to setup your ssh environment in /volume1/git_repo/.ssh/environment location instead of /root/.ssh/environment.

Once I changed this, the git PATH issues resolved and I was able to clone the repo from a different PC using ssh.

To test if you ssh environment changes worked as expected, try the command -

ssh username@ipaddress env

This should print all the environment variables and in there you should see your PATH variable listed as expected.

Hope this helps!

-Vijay

djfil said...

Thanks.

And to the commenters too!

This helped me a lot installing git with opkg

Jarod51 said...

Hi

A little offroad subject. Anyone has managed to install symfony2 (which uses git submodules a lot) on a syno ?

BlogSnot said...

fwbackup Question
Thanks to everyone who has contributed here, awesome stuff. I hope someone here might be able to figure this out:
I am able to SSH into my Syno DS211J NAS from my Linux Mint 12 box with no trouble. However, in trying to setup fwbackups to perform remote backups using SSH (to the Syno DS211J NAS), I get a "Error: Channel closed" msg.

I'm sure my settings are correct (IP address, port #22, username, password, and folder). If anyone has encountered this problem and has a solution, it would be greatly appreciated.

jeroen_peelaerts said...

44 down vote


You can also use the "-u" option to specify the path. I find this helpful on machines where my .bashrc doesn't get sourced in non-interactive sessions. For example,

git clone -u /home/you/bin/git-upload-pack you@machine:code

jeroen_peelaerts said...

Also, the upload-pack path can be set permanently by running the following commands after cloning, which eliminates the need for --upload-pack on subsequent pull/fetch requests. Similarly, setting receive-pack eliminates the need for --receive-pack on push requests.

git config remote.origin.uploadpack /path/to/git-upload-pack
git config remote.origin.receivepack /path/to/git-receive-pack

alishka said...

For me on DSM3.2
/usr/syno/etc/rc.d/S95sshd.sh restart
didn't restart sshd, only killed currently running sessions. In the end I restarted it via web GUI and it reread the config.

othbert said...

I had the same problems with restarting the ssh daemon, however, even after bootin' the whole NAS, the ~/.ssh/environment files weren't being read. Strangely enough when I created a ~/.profile file for the user, based on the /root/.profile, it finally worked great!

Thanks for the great write up!

ForgotMyOrange said...
This comment has been removed by the author.
Harry Disseldorp said...

Hi all,
I had similar problem on my DS712+ running DSM 4.2-3211.
None of the suggested options did work to my satisfaction.
The problem was, that when I tried to clone a git repository from the DS to my local machine the following error message appeared:
'sh: git-upload-pack: not found'
I tried all the above suggestions related to this message. None of them (except one) was the solution to my problem. The -u option did work, but only on the command line. It did NOT work from out of an application like PHPStorm. The solution which finally worked is as follows: I created 2 symbolic links.
ln -s /opt/bin/git-upload-pack /usr/bin/git-upload-pack
ln -s /opt/bin/git-receive-pack /usr/bin/git-receive-pack
The reason I came to this solution is I read in the sshd_config file that the ssh version was compiled with PATH=/usr/bin:/usr/sbin .... etc.
I assume that ssh on the DS machine will never look outside the precompiled scope. Even when the .ssh/environment exists and is enabled. And regardless which shell has been specified in the /etc/passwd.

Ronnie Stevens said...

I had the same error:

error: cannot run /opt/bin/ssh: No such file or directory
fatal: unable to fork

So I checked where my ssh was:

> which ssh
/opt/bin/ssh

then as root:
> cd /opt/bin
> ln -s /usr/syno/bin/ssh .

thus creating a symlink from the working ssh to the directory git is looking

Post a Comment