Monday, February 20, 2012

F5 Pacfiles R Us Part Duex

 

In my previous article I explained how pac files can be consolidated by using an F5 Load Balancer eliminating a web server in the process.   The pac file for the most part was very static.  What if you want to make it dynamic – such that you have many different proxy systems and you want to serve a different version of the pacfile based on client’s source address.

In this article I am going to show you a way to make it dynamic in 2 steps

 

1. First you need to create a data group contain the IP address of the clients associated to the proxy systems that you want to use.

Name: regions_proxy

Type: Address

"10.10.10.0/255.255.255.0" := "proxy-a.domain.com:8080"

"10.10.0.0/255.255.0.0" := "proxy-b.domain.com:8080"

.

.

.

"10.0.0.0/255.0.0.0" := "proxy-c.domain.com:8080"

...Etc

 

 

2.    Then you need the iRule itself.  This time you are going to use Datagroups and variables

when RULE_INIT {
        # Set the contents of the PAC file to be delivered within static::pacfile. While
        # specific logic here is fine, the "localized" proxy
        # should be returned using the $selected_proxy variable... this
        # variable will be filled in when the file delivered
        # with the value learned from the DataGroup.

   set static::pacfile {
     function FindProxyForURL(url, host) {

         if (isPlainHostName(host))
         return "DIRECT";

         if (shExpMatch(url, "http://10.*")||
         shExpMatch(url, "https://10.*")||
         shExpMatch(url, "ftp://10.*")||
         shExpMatch(url, "http://localhost*")||
         shExpMatch(url, "https://localhost*")||
         shExpMatch(url, "http://127.0.0.1*")||
         shExpMatch(url, "https://127.0.0.1*")||
         shExpMatch(url, "http://172.*")||
         shExpMatch(url, "https://172.*")||
         shExpMatch(url, "ftp://172.*"))
         return "DIRECT";

      if (dnsDomainIs(host, ".extranet.com")||
         dnsDomainIs(host, ".extranet2.com"))
         return "Proxy $proxyselect";
        
         if (dnsDomainIs(host, ".intrant.com")||
         dnsDomainIs(host, ".intranet2.com"))
         return "DIRECT";
       
      return "PROXY $proxyselect";
      }
   }
}

when CLIENT_ACCEPTED {

        # Create a DataGroup class called "proxy_regions" and populate it with
        # the IP networks and their proxy value assignments:
        #
        # "10.0.0.0/8" := "proxya.domain.com:8080"

        if { [class match [IP::client_addr] eq regions_proxy] } {
                set proxyselect "[class match -value [IP::client_addr] eq regions_proxy]"
        } else {
                set proxyselect "DIRECT"
        }
}


when HTTP_REQUEST {
   # Returns pacfile via "proxy.pac" as part of the HTTP Request
   # with specific proxy Content-type
   switch [HTTP::uri] {
      "/proxy.pac" {
         HTTP::respond 200 content [subst $static::pacfile] "Content-Type" "application/x-ns-proxy-autoconfig" "pragma" "no-cache"
      }
   }
}

As you can see with datagroups and variables.  You can control how the pacfile is structured and customized without create different file versions.

F5 Pacfiles R Us

As some of my friends would agree I am an F5 evangelist.  The reason why I am such a fan is because it continues to eliminate or lesson some of the biggest head-aches that I experience at work. One of those pain points that it helps lesson is managing and using pac files.

Pac Files are used throughout various companies to direct traffic out proxy systems but is not wel understood by most and how it relates to F5 ADCs.  Hopefully this article can clear it up for you and give you a better how idea of how to use it with the ADC.

What are pac file?  A pac file is a text file containing JavaScript code.  It is commonly called proxy.pac, but it can be named differently. It’s mostly used for 2 primary reasons:

  • Companies have multiple proxies where traffic is directed through, either for redundancy or access to networks that are normally closed off.
  • Split domain, where a single domain (i.e., domain.com ) is hosted both internally and externally.

A very basic pac file contains a single function.  It’s function is to direct users through a proxy or proxies based on a URL or website request.   A more advanced pacfile script can direct users out through a proxy or proxies based on user’s IP address or subnet.

The pac file settings are usually the following link  http://pac.webserver.com/proxy.pac.  Pac files can also be hosted on different ports where the links for eample port 7070 (i.e., http://pac.webserver.com:7070/proxy.pac)

The pac file setting is configured in the browsers option or preference settings.   Most cases they are labeled clearly.

The pac file can be hosted on the user’s PC or it can be hosted on a web server (i.e. Apache or IIS). Most companies host in a web server, for centralized control.  It’s important to remember that, so far no browser supports more then one pacfile at the user browser level.  However, there are certain 3rd party extensions that can be used to get around this limitation.  However, it’s usually designed for a specific browser rather then a universal version.

 

So now you understand what a pac file is, what does a standard syntax look like?

 

  1: function FindProxyForURL(url, host) {
  2:     
  3:     
  4:     if (shExpMatch(url, "http://10.*")||
  5:     shExpMatch(url, "https://10.*")||
  6:     shExpMatch(url, "ftp://10.*")||
  7:     shExpMatch(url, "http://localhost*")||
  8:     shExpMatch(url, "https://localhost*")||
  9:     shExpMatch(url, "http://127.0.0.1*")||
 10:     shExpMatch(url, "https://127.0.0.1*")||
 11:     shExpMatch(url, "http://172.*")||
 12:     shExpMatch(url, "https://172.*")||
 13:     shExpMatch(url, "ftp://172.*"))
 14:     return "DIRECT";
 15:         
 16: 
 17:     if (dnsDomainIs(host, "insidesite.com")||
 18:     dnsDomainIs(host, "servername.google.com"))
 19:     return "DIRECT";
 20: 
 21:   return "PROXY proxy.internaldomain.com:8080";
 22:   }
 23: }

 

Simple?  It’s actually simple once you know what each lines are doing.   Let me explain: Lines 4 to 14 state that if a user entered a HTTP request that’s a private address or a localhost or loopback,  should go direct and not use a proxy.   Lines 17 to 19 state that if a user entered a HTTP request where the hostname matches either those 2 entries - should go direct and not use a proxy.  Line 21 is the catch all – if the user’s HTTP request does not match any of the IF conditional statements, the script forces a request to go to proxy.internaldomain.com on port 8080.   It’s important to note that the pacfile scripts are processed at the client and NOT on the server that is delivering the pacfile.

 

Now that you the basic idea of pac file syntax, the next step is to know how it works with respect to user, proxy and a web server holding a webpage.

Let’s say the Pac file is hosted on a proxy server also running a web server.  Let’s assume that the user already has the proxy pac file settings already configured in their browser.

 

clip_image002

Step 1: When the user starts the internet browser, browser retrieves the pac file .

 

 

clip_image003

Step 2: The pacfile is downloaded to the browser’s cache and process any URL requests (For example the user’s browser could have home page going to www.cnn.com) before it let’s the user enter information in the URL

 

Step 3: The user then enters in the URL http://www.google.com

 

clip_image005

Step 4:  The browser runs through the pacfile script  like the one above, attempting to match to a IF conditional statement.   In our example it won’t match any IF statement statement – thus by default is sent to “PROXY proxy.internaldomain.com:8080”

 

clip_image006

Step 5: Proxy makes a request to the internet web server and requests for www.google.com page on behalf of the user.

 

clip_image007

Step 6: The page is returned to the proxy.

 

clip_image008

 

Step 7: As a final step, the proxy sends the request web page to user.

 

Okay this is great.  What does this have to do with ADCs?  As it turns out there is one disadvantage for pacfiles – hosting a pacfile.  It requires a web server.  While the thought of hosting a web page might not be earth shattering it does become a problem when you run into the following scenario.

  • Your IT organization might mandate a reduction in the amount of web servers in the environment.
  • Your IT organization could contain a mixture of Squid, Bluecoat, and/or Ironport proxy systems.  Each proxy has their own distinctive method to deliver a pac file, some might provide you flexibility to control your pac file deliver and some might not.
  • Your IT organization wants to deliver different pac files based on users location or type of browser or IP addresses.

 

What if you can host the pac file in an iRule?    A centralized and consistent approach to deliver a pac file(s) to a user without requiring a web server OR relying on proxy vendors.

 

The iRule

IMPORTANT NOTE: From this point the iRule is written to conform with v10

So how do we get an iRule to deliver a pacfile?

 

At this point we know the following:

  • Pac file is a JavaScript code that the browsers understands
  • Pac file is delivered via HTTP REQUEST and RESPONSE

 

function FindProxyForURL(url, host) {
	if (isPlainHostName(host))
	return "DIRECT";

	if (shExpMatch(url, "http://10.*")||
	shExpMatch(url, "https://10.*")||
	shExpMatch(url, "ftp://10.*")||
	shExpMatch(url, "http://localhost*")||
	shExpMatch(url, "https://localhost*")||
	shExpMatch(url, "http://127.0.0.1*")||
	shExpMatch(url, "https://127.0.0.1*")||
	shExpMatch(url, "http://172.*")||
	shExpMatch(url, "https://172.*")||
	shExpMatch(url, "ftp://172.*"))
	return "DIRECT";		
	if (dnsDomainIs(host, "inside.com")||
	dnsDomainIs(host, "inside.net")||
	dnsDomainIs(host, "outside.net"))
	return "DIRECT";
 
     return "PROXY proxy.internaldomain.com:8080";
	}
}

 

What’s the first step?   Since the script needs to be human readable with indents and spaces we need to set the entire Pac file script into a global variable for example “pacfile”

 

  1: 	set static::pacfile {
  2: 	function FindProxyForURL(url, host) {
  3: 		
  4: 		if (isPlainHostName(host))
  5: 		return "DIRECT";
  6: 	  
  7: 		if (shExpMatch(url, "http://10.*")||
  8: 		shExpMatch(url, "https://10.*")||
  9: 		shExpMatch(url, "ftp://10.*")||
 10: 		shExpMatch(url, "http://localhost*")||
 11: 		shExpMatch(url, "https://localhost*")||
 12: 		shExpMatch(url, "http://127.0.0.1*")||
 13: 		shExpMatch(url, "https://127.0.0.1*")||
 14: 		shExpMatch(url, "http://172.*")||
 15: 		shExpMatch(url, "https://172.*")||
 16: 		shExpMatch(url, "ftp://172.*"))
 17: 		return "DIRECT";
 18: 				
 19: 
 20: 		if (dnsDomainIs(host, "inside.com")||
 21: 		dnsDomainIs(host, "inside.net")||
 22: 		dnsDomainIs(host, "outside.net"))
 23: 		return "DIRECT";
 24: 
 25: 	return "PROXY proxy.internaldomain.com:8080";
 26: 	}
 27: }
 28: 

 

The event to use is when RULE_INIT since we are setting the script into a global variable.  Your code looks like the following

  1: when RULE_INIT {
  2: 	set static::pacfile {
  3: 	function FindProxyForURL(url, host) {
  4: 		
  5: 		if (isPlainHostName(host))
  6: 		return "DIRECT";
  7: 	  
  8: 		if (shExpMatch(url, "http://10.*")||
  9: 		shExpMatch(url, "https://10.*")||
 10: 		shExpMatch(url, "ftp://10.*")||
 11: 		shExpMatch(url, "http://localhost*")||
 12: 		shExpMatch(url, "https://localhost*")||
 13: 		shExpMatch(url, "http://127.0.0.1*")||
 14: 		shExpMatch(url, "https://127.0.0.1*")||
 15: 		shExpMatch(url, "http://172.*")||
 16: 		shExpMatch(url, "https://172.*")||
 17: 		shExpMatch(url, "ftp://172.*"))
 18: 		return "DIRECT";
 19: 				
 20: 
 21: 		if (dnsDomainIs(host, "inside.com")||
 22: 		dnsDomainIs(host, "inside.net")||
 23: 		dnsDomainIs(host, "outside.net"))
 24: 		return "DIRECT";
 25: 
 26: 	return "PROXY proxy.internaldomain.com:8080";
 27: 	}
 28: 	}
 29: }

 

The next step is to use HTTP_REQUEST event as the trigger for the request  and use HTTP::response command to send the the contents of the pacfile back to the requestor.   We also need to include a MIME type so the browser knows that proxy pac file is being send.  This is a requirement.  We also need to make sure that requestor’s browser does not cache the file for any length of time.   This is because we want to make updates and get immediate changes to the requestor.

Based on this the code should look something like the following

 

  1: when HTTP_REQUEST {
  2: 	switch -glob [HTTP::uri] {
  3: 	  "/proxy.pac" {
  4: 		HTTP::respond 200 content $static::pacfile "Content-Type" "application/x-ns-proxy-autoconfig" "pragma" "no-cache"
  5: 		return
  6: 	               }
  7: 	}
  8: }
 

 

Putting it all together the entire code should look like the following

  1: when RULE_INIT {
  2:   set static::pacfile {
  3:   function FindProxyForURL(url, host) {
  4:     
  5:     if (isPlainHostName(host))
  6:     return "DIRECT";
  7:     
  8:     if (shExpMatch(url, "http://10.*")||
  9:     shExpMatch(url, "https://10.*")||
 10:     shExpMatch(url, "ftp://10.*")||
 11:     shExpMatch(url, "http://localhost*")||
 12:     shExpMatch(url, "https://localhost*")||
 13:     shExpMatch(url, "http://127.0.0.1*")||
 14:     shExpMatch(url, "https://127.0.0.1*")||
 15:     shExpMatch(url, "http://172.*")||
 16:     shExpMatch(url, "https://172.*")||
 17:     shExpMatch(url, "ftp://172.*"))
 18:     return "DIRECT";
 19:         
 20:     if (dnsDomainIs(host, "inside.com")||
 21:     dnsDomainIs(host, "inside.net")||
 22:     dnsDomainIs(host, "outside.net"))
 23:     return "DIRECT";
 24: 
 25:   return "PROXY proxy.internaldomain.com:8080";
 26:   }
 27:   }
 28: }
 29: 
 30: when HTTP_REQUEST {
 31:   switch -glob [HTTP::uri] {
 32:     "/proxy.pac" {
 33:       HTTP::respond 200 content $static::pacfile "Content-Type" "application/x-ns-proxy-autoconfig" "pragma" "no-cache"
 34:       return
 35:     }
 36:   }
 37: }

As you can see you now have the ability to deliver a pac file without using a web server

 

Tuesday, January 17, 2012

Custom Commercial Home Video Streaming

A colleague of mine came to me and asked me "How can I stream my home videos or any digital video I have to the big screen in a way that doesn't require me to be an expert on computers."  So I wanted to share with a simple setup outline that I have tried as well who have tested this and worked for them.


Here is what you need at the minimum:
Hardware
- HDTV (Any will do)
- Apple TV 2 (IOS 4.3 or above)
- iPod or iPAD or iPhone (IOS v4.3 and up). 
- PC / MAC with at least 200GB of storage with decent processor (a good comparison is 2011 Mac Book Pro which has enough computing power)


**Software
- Air Video Server
- Air Video Mobile (For the iPod/iPAD/iPhone)


With these components you can stream pretty much any video to the big screen.


Here is the basic setup:


Assumption: You need a home wifi that is supporting 802.11n (5Ghz) .  You will need dual band wifi (802.11g/n) if your iPod or iPhone only supports 802.11g


1. Setup your AppleTV hooked up to your home Wifi on 802.11n and AirPlay turned on.
2. Next you will need a PC/MAC where you store your video.  You download AirVideo server (http://www.inmethod.com/air-video/download.html).  You install, run the software and point to the directory where you store you video.
3. You download Air Video Mobile on your iPod. (You can download this http://www.inmethod.com/air-video/index.html)
NOTE: Your iPOD will use the 2Ghz frequency for wireless hookup. Why?


From this point on you can view the video on the iPod through Air Video App.  When you want to view this on your HDTV then you use Airplay to point the stream to the Apple TV (You can view use or Airplay here http://support.apple.com/kb/HT4437).   


A couple of things you need to keep in mind:
- You can also control forwarding, skip, rewind through your iPod which essentially services as your remote control for streaming your video.
- Transcoding is done at the PC/MAC that contains the AirVideo server

This is it in a nutshell. I haven't seen any of my other collegues really have issues with this setup. In most cast they already had the hardware, but only needed the Air Video software.

I hope this little outline helps



Monday, January 16, 2012

REFERENCES: F5 TCPDUMP for Dummies

This month I received about 2 or 3 requests from my clients to educate them on TCPDUMP with respect to the F5.  There are plenty of TCPDUMP HOW-TO's (a popular one can be found here).   This is slightly the same concept as a Dummies book, but more or less a FAQ.
I hope you like the article

What is TCPDUMP?
The tcpdump utility is a command line packet sniffer with many features and options. For a full description, refer to the tcpdump man pages by typing the following command:
man tcpdump


Q: What devices use TCPDUMP?
A: It is available on the F5 BIGIP Devices as well Linux devices.
Q: Selecting an interface or VLAN?
A: The tcpdump utility is able to sniff for packets on only one interface or VLAN. By default, it will select the lowest numbered interface.
To select an interface, use the -i flag as follows:
tcpdump -i <interface>
Examples:
tcpdump -i exp1
tcpdump -i 1.10
tcpdump -i internal

Q: How do you disable name resolution?
A:By default, tcpdump will attempt to look up IP addresses and use names, rather than numbers, in the output. BIG-IP must wait for a response from the DNS server, so the lookups can be time consuming and the output may be confusing.
To disable name resolution, use the -n flag as in the following examples:
tcpdump -n
tcpdump -ni internal
Q: How do you save TCPDUMP output to a file?
You can save the tcpdump data to one of the following file formats:
  • A binary file that contains all the information collected by the tcpdump and is readable by the tcpdump utility as well as many other traffic analysis packages.
  • A text file that contains a subset of the full tcpdump data, but is readable only as plain text.
Binary file
To save the tcpdump output to a binary file, type the following command:
tcpdump -w <filename>
For example:
tcpdump -w dump1.bin
Note: The tcpdump utility will not print data to the screen while it is capturing to a file. To stop the capture, press CTRL-C.
Text file
To save the tcpdump output to a text file, type the following command:
tcpdump > filename.txt
For example:
tcpdump > dump1.txt

Q: How do you read binary file output?
A:To read data from a binary tcpdump file (that you saved by using the tcpdump -w command), type the following command:
tcpdump -r <filename>
For example:
tcpdump -r dump1.bin
In this mode, the tcpdump utility reads stored packets from the file, but otherwise operates just as it would reading from the network interface. As a result, you can use formatting commands and filters.
Q: How do I do filtering?
A:The tcpdump utility allows you to use filters to, among other things, restrict the output to specified addresses and ports and specified tcp flags.
Filtering on a host address
  • To view all packets that are traveling to or from a specific IP address, type the following command:
tcpdump host IP_ADDRESS
For example:
tcpdump host 10.90.100.1
  • To view all packets that are traveling from a specific IP address, type the following command:
tcpdump src host IP_ADDRESS
For example:
tcpdump src host 10.90.100.1
  • To view all packets that are traveling to a particular IP address, type the following command:
tcpdump dst host IP_ADDRESS
For example:
tcpdump dst host 10.90.100.1
Filtering on a port
  • To view all packets that are traveling through the BIG-IP system and are either sourced from or destined to a specific port, type the following command:
tcpdump port PORT_NUM
For example:
tcpdump port 80
  • To view all packets that are traveling through the BIG-IP system and sourced from a specific port, type the following command:
tcpdump src port PORT_NUM
For example:
tcpdump src port 80
  • To view all packets that are traveling through the BIG-IP system and destined to a specific port, type the following command:
tcpdump dst port PORT_NUM
For example:
tcpdump dst port 80
Filtering on a tcp flag
  • To view all packets that are traveling through the BIG-IP system that contain the SYN flag, type the following command:
tcpdump 'tcp[tcpflags] & (tcp-syn) != 0'
  • To view all packets that are traveling through the BIG-IP system that contain the RST flag, type the following command:
tcpdump 'tcp[tcpflags] & (tcp-rst) != 0'
Combining filters with the and operator
You can use the and operator to filter for a mixture of output.
Following are some examples of useful combinations:
tcpdump host 10.90.100.1 and port 80
tcpdump src host 172.16.101.20 and dst port 80
tcpdump src host 172.16.101.20 and dst host 10.90.100.

Q: How do I combine TCPDUMP options?
A: This Solution contains the most essential tcpdump options. You will generally need to use most of the options in combination.
Following are examples of how to combine the tcpdump options to provide the most meaningful output:
tcpdump -ni internal -w dump1.bin
tcpdump -ni internal -r dump1.bin host 10.90.100.1
tcpdump -ni exp1 host 10.90.100.1 and port 80
tcpdump -ni 1.10 src host 172.16.101.20 and dst port 80 >dump1.txt

COMMANDS: SSH, not so obvious uses

SSH is used practically every day by various types of administrators and applications. SSH is one of the the most documented commands.  So what I have done is looked for commands that use SSH for some not so obvious uses.  Hopefully this help or inspire you to use it for whatever solution you want to provide

1) Copy ssh keys to user@host to enable password-less ssh logins.
ssh-copy-id user@host
To generate the keys use the command ssh-keygen


2) Start a tunnel from some machine’s port 80 to your local post 2001
ssh -N -L2001:localhost:80 somemachine
Now you can acces the website by going to http://localhost:2001/


3) Output your microphone to a remote computer’s speaker
dd if=/dev/dsp | ssh -c arcfour -C username@host dd of=/dev/dsp
This will output the sound from your microphone port to the ssh target computer’s speaker port. The sound quality is very bad, so you will hear a lot of hissing.

4) Compare a remote file with a local file
ssh user@host cat /path/to/remotefile | diff /path/to/localfile
Useful for checking if there are differences between local and remote files.


5) Mount folder/filesystem through SSH
sshfs name@server:/path/to/folder /path/to/mount/point
Install SSHFS from http://fuse.sourceforge.net/sshfs.html
Will allow you to mount a folder security over a network.


6) SSH connection through host in the middle
ssh -t reachable_host ssh unreachable_host
Unreachable_host is unavailable from local network, but it’s available from reachable_host’s network. This command creates a connection to unreachable_host through “hidden” connection to reachable_host.


7) Copy from host1 to host2, through your host
ssh root@host1 “cd /somedir/tocopy/ && tar -cf – .” | ssh root@host2 “cd /samedir/tocopyto/ && tar -xf -”
Good if only you have access to host1 and host2, but they have no access to your host (so ncat won’t work) and they have no direct access to each other.


8) Run any GUI program remotely
ssh -fX <user>@<host> <program>
The SSH server configuration requires:
X11Forwarding yes # this is default in Debian
And it’s convenient too:
Compression delayed

9) Create a persistent connection to a machine
ssh -MNf <user>@<host>
Create a persistent SSH connection to the host in the background. Combine this with settings in your ~/.ssh/config:
Host host
ControlPath ~/.ssh/master-%r@%h:%p
ControlMaster no
All the SSH connections to the machine will then go through the persisten SSH socket. This is very useful if you are using SSH to synchronize files (using rsync/sftp/cvs/svn) on a regular basis because it won’t create a new socket each time to open an ssh connection.


10) Attach screen over ssh
ssh -t remote_host screen -r
Directly attach a remote screen session (saves a useless parent bash process)


11) Port Knocking!
knock <host> 3000 4000 5000 && ssh -p <port> user@host && knock <host> 5000 4000 3000
Knock on ports to open a port to a service (ssh for example) and knock again to close the port. You have to install knockd.
See example config file below.
[options]
logfile = /var/log/knockd.log
[openSSH]
sequence = 3000,4000,5000
seq_timeout = 5
command = /sbin/iptables -A INPUT -i eth0 -s %IP% -p tcp –dport 22 -j ACCEPT
tcpflags = syn
[closeSSH]
sequence = 5000,4000,3000
seq_timeout = 5
command = /sbin/iptables -D INPUT -i eth0 -s %IP% -p tcp –dport 22 -j ACCEPT
tcpflags = syn


12) Remove a line in a text file. Useful to fix
ssh-keygen -R <the_offending_host>
In this case it’s better do to use the dedicated tool


13) Run complex remote shell cmds over ssh, without escaping quotes
ssh host -l user $(<cmd.txt)
Much simpler method. More portable version: ssh host -l user “`cat cmd.txt`”


14) Copy a MySQL Database to a new Server via SSH with one command
mysqldump –add-drop-table –extended-insert –force –log-error=error.log -uUSER -pPASS OLD_DB_NAME | ssh -C user@newhost “mysql -uUSER -pPASS NEW_DB_NAME”
Dumps a MySQL database over a compressed SSH tunnel and uses it as input to mysql – i think that is the fastest and best way to migrate a DB to a new server!


15) Remove a line in a text file. Useful to fix “ssh host key change” warnings
sed -i 8d ~/.ssh/known_hosts


16) Copy your ssh public key to a server from a machine that doesn’t have ssh-copy-id
cat ~/.ssh/id_rsa.pub | ssh user@machine “mkdir ~/.ssh; cat >> ~/.ssh/authorized_keys”
If you use Mac OS X or some other *nix variant that doesn’t come with ssh-copy-id, this one-liner will allow you to add your public key to a remote machine so you can subsequently ssh to that machine without a password.


17) Live ssh network throughput test
yes | pv | ssh $host “cat > /dev/null”
connects to host via ssh and displays the live transfer speed, directing all transferred data to /dev/null
needs pv installed
Debian: ‘apt-get install pv’
Fedora: ‘yum install pv’ (may need the ‘extras’ repository enabled)


18) How to establish a remote Gnu screen session that you can re-connect to
ssh -t user@some.domain.com /usr/bin/screen -xRR
Long before tabbed terminals existed, people have been using Gnu screen to open many shells-  a single text terminal. Combined with ssh, it gives you the ability to have many open shells with a single remote connection using the above options. If you detach with “Ctrl-a d” or if the ssh session is accidentally terminated, all processes running in your remote shells remain undisturbed, ready for you to reconnect. Other useful screen commands are “Ctrl-a c” (open new shell) and “Ctrl-a a” (alternate between shells). Read this quick reference for more screen commands: http://aperiodic.net/screen/quick_reference


19) Resume scp of a big file
rsync –partial –progress –rsh=ssh $file_source $user@$host:$destination_file
It can resume a failed secure copy ( usefull when you transfer big files like db dumps through vpn ) using rsync.
It requires rsync installed in both hosts.
rsync –partial –progress –rsh=ssh $file_source $user@$host:$destination_file local -> remote
or
rsync –partial –progress –rsh=ssh $user@$host:$remote_file $destination_file remote -> local


20) Analyze traffic remotely over ssh w/ wireshark
ssh root@server.com ‘tshark -f “port !22″ -w -’ | wireshark -k -i
This captures traffic on a remote machine with tshark, sends the raw pcap data over the ssh link, and displays it in wireshark. Hitting ctrl+C will stop the capture and unfortunately close your wireshark window. This can be worked-around by passing -c # to tshark to only capture a certain # of packets, or redirecting the data through a named pipe rather than piping directly from ssh to wireshark. I recommend filtering as much as you can in the tshark command to conserve bandwidth. tshark can be replaced with tcpdump thusly:
ssh root@example.com tcpdump -w – ‘port !22′ | wireshark -k -i


21) Have an ssh session open forever
autossh -M50000 -t server.example.com ‘screen -raAd mysession’
Open a ssh session opened forever, great on laptops losing Internet connectivity when switching WIFI spots.


22) Harder, Faster, Stronger SSH clients
ssh -4 -C -c blowfish-cbc
We force IPv4, compress the stream, specify the cypher stream to be Blowfish. I suppose you could use aes256-ctr as well for cypher spec. I’m of course leaving out things like master control sessions and such as that may not be available on your shell although that would speed things up as well.


23) Throttle bandwidth with cstream
tar -cj /backup | cstream -t 777k | ssh host ‘tar -xj -C /backup’
this bzips a folder and transfers it over the network to “host” at 777k bit/s.
cstream can do a lot more, have a look http://www.cons.org/cracauer/cstream.html#usage
for example:
echo w00t, i’m 733+ | cstream -b1 -t2


24) Transfer SSH public key to another machine in one step
ssh-keygen; ssh-copy-id user@host; ssh user@host
This command sequence allows simple setup of (gasp!) password-less SSH logins. Be careful, as if you already have an SSH keypair in your ~/.ssh directory on the local machine, there is a possibility ssh-keygen may overwrite them. ssh-copy-id copies the public key to the remote host and appends it to the remote account’s ~/.ssh/authorized_keys file. When trying ssh, if you used no passphrase for your key, the remote shell appears soon after invoking ssh user@host.


25) Copy stdin to your X11 buffer
ssh user@host cat /path/to/some/file | xclip
Have you ever had to scp a file to your work machine in order to copy its contents to a mail? xclip can help you with that. It copies its stdin to the X11 buffer, so all you have to do is middle-click to paste the content of that looong file :)