Monitor Servers Behind a Firewall Using Reverse SSH Tunnels (and Cacti)

Wow. What a subject line! It’s a little technical magic, but fairly easy to do. I won’t cover installing Cacti or the SNMP daemon. There are plenty of guides for those. However, guides for creating and using the reverse SSH tunnel automagically are a little scattered, so I decided to document it for myself. This is all using Ubuntu 10.04 LTS server with Cacti 0.8.7e. I use daemontools to help keep the SSH tunnels up. Any references to “local” means the server that initiates the reverse SSH tunnel (usually behind a firewall). References to “remote” are the server running Cacti that can’t reach the “local” server(s) behind a firewall.

[UPDATE] I figured out how to do it for Windows! I’ve added it down at the bottom. [/UPDATE]

For *nix
On local, generate an SSH key. Do not enter a password. (I haven’t figured out how to make it automagically happen with ssh-agent yet).

sudo -u snmp ssh-keygen -t rsa

Local: copy the contents of /var/lib/snmp/.ssh/id_rsa.pub

sudo cat /var/lib/snmp/.ssh/id_rsa.pub

Remote: add the contents of id_rsa.pub above the authorized_keys

sudo pico -w /var/lib/snmp/.ssh/authorized_keys

Remote: prepend the following to the key you just added in authorized_keys. You may want to change the from option to include your local subnet/domain that your local machines are on. The command option makes sure that you really can’t do anything if someone unauthorized got access to the account.

from="*.epbfi.com",command="/bin/false",no-X11-forwarding,no-pty 

Local: install and start daemontools

1
2
sudo apt-get install daemontools-run
sudo start svscan

Local: create the daemontools service directory

sudo mkdir /etc/service/snmp-ssh-tunnel

Local: create the daemontools run script and set it to executable. Feel free to change the 16101 port and remote.server.tld to reflect your own remote port and server DNS name or IP address. I chose 1610x to reflect the usual SNMP port number plus a couple extra digits. *shrug* It’s all up to you!

1
2
sudo echo -e '#!/bin/sh\n\nsetuidgid snmp ssh -nNT -R 16101:localhost:161 snmp@remote.server.tld' | sudo tee - /etc/service/snmp-ssh-tunnel/run
sudo chmod +x /etc/service/snmp-ssh-tunnel/run

That’s pretty much it for the technical stuff. Now, when you set up a host in Cacti, just use “tcp:localhost” for the Hostname field and put in the port number (in this case 16101) in the SNMP Port field.

It worked for me, so hopefully it will work for anyone else!

For Windows
It’s almost the same as the *nix setup. You’re creating a reverse SSH tunnel just the same. However, we’re just using the help of Net-SNMP to proxy the TCP SNMP from the SSH tunnel to the Windows UDP SNMP agent.

Install the windows SNMP agent (Simple Network Management Protocol). It’s usually under “Add/Remove Programs” and/or “Programs and Features”. Look for something along the lines of Windows Features. It changes for each version of Windows. I’m pretty sure it works for any version of Windows for and after 2000/XP.

Download and install Net-SNMP from http://www.net-snmp.org. You have to install it to the default C:\usr directory. It seems the scripts are hard-coded. The standard agent installation is fine. Follow any other recommended installation instructions. Just don’t do anything beyond that. No need to even try running snmpconf.

Download and install ActiveState’s ActivePerl (the free community edition) for Windows from http://www.activestate.com. Once again, follow any recommended installation instructions.

Download and install BitVise Tunnelier from http://www.bitvise.com/tunnelier. Follow any recommended installation instructions.

Create a C:\usr\etc\snmp\snmpd.conf text file and put the following in it. Just be sure to replace the “XXXXX” with your community name.

rocommunity XXXXX
proxy 127.0.0.1
agentaddress tcp:127.0.0.1:161

Open a command prompt and run “snmpd -register”. This will register snmpd as a Windows service. If you’re running Vista/7/2008, it’ll probably come up with some of application crash error. Just click OK, close, or whatever to make it go away. Go to Windows Services and start the Net-SNMP Agent.

Now we need to get the tunnel set up. Launch Tunnelier. Enter the hostname, port, and username. Leave the Initial method as “none”. Now click on the “User keypair manager” under the authentication section. Click the “Generate New” button. Leave the password blank and the other defaults should be slot 1, ssh-rsa algorithm, 2048 bits, and enter something that will help identify this key in the comments. Click Generate and it will take you back to the keypair manager. Select the key and click Export. Leave “Export Public Key” checked, select “OpenSSH format” and click Export. This will ask you for a location to save the file. Put it anywhere you’re comfortable (your desktop is easiest) with and give it a “.txt” extension. Do the export process again except select “Export private key” and “Bitvise format” and store it somewhere convenient. Now, close the keypair manager and click Login. It will ask you to verify the host key. Click “Accept and Save” and then “Cancel” on the next popup. We just wanted to get the host key. Click on the “Host key manager” link under the server/port fields and export the host key in Bitvise format. Close the host key manager and exit out of Bitvise.

Open the key file that you exported earlier (with the “.txt” extension) and copy and paste that key into the authorized_key files just as you did (or would have done) in the *nix instructions above. You can delete this file now. Create a new directory somewhere on your local computer (I used C:\Tunnelier) and copy the private key and host key files to that directory.

And, finally, you will want to set up a scheduled task to run when your user is logged in to the Windows box. I can’t seem to get this running in any sort of daemon mode even after consulting with Bitvise’s help regarding running it as a service. So, for now, I’ll just stick with the run-on-login method. If you need help setting up some sort of auto-login, go to Start -> Run and then type in “control userpasswords2″. Select the username and enter the password and you’ll be able to auto-login. If this lacks certain security measures for you, you could set up a screensaver to run at 1 minute of idle time and enable password-protection when it exits. Oh, right, the scheduled task. Create a new (basic) scheduled task that will run on login. The path to executable is the full path to Tunnelier.exe. In my case, this is “C:\Program Files (x86)\Bitvise Tunnelier\Tunnelier.exe”. You may have to include the quotes to make sure Windows will recognize the full path. The arguments will be as follows (changing the appropriate paths, hosts, usernames, ports, etc as needed).

-host=remote.host.tld -user=snmp -s2c=127.0.0.1,16102,127.0.0.1,161 -loginOnStartup -reExecOnReconn=y  -noRegistry -keypairFile="C:\Tunnelier\snmp-ssh-tunnel.key" -hostKeyFile="C:\Tunnelier\snmp-ssh-tunnel.hostkey" -proxy=n -openTerm=n -openSFTP=n -openRDP=n -exitOnLogout

Now, start your scheduled task (you have to start it this first time) and check for the remote connection. Do an snmpwalk to test and you should be ready to go.

This entry was posted in Geek Stuff and tagged . Bookmark the permalink.

2 Responses to Monitor Servers Behind a Firewall Using Reverse SSH Tunnels (and Cacti)

  1. Neil says:

    nice article! instead of deamontools, is it possible to run

    ssh -nNT -R 16101:localhost:161 snmp@remote.server.tld

    as a cron job?

  2. Garrett says:

    The difference between a program like daemontools and crontab is that crontab will run the command in regular intervals whereas daemontools only runs the command once and monitors the process afterwards. It’s possible to set up a wrapper script that checks if the process is still running and have crontab run that script every few minutes.

    I chose daemontools mostly because we’ve been using it at work for the past 10 years or so. There are probably alternatives to daemontools out there, but daemontools fit the bill for me.

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <pre lang="" line="" escaped="" highlight="">

Spam protection by WP Captcha-Free