[dba-Tech] windows subsystem for Linux (WSL)

[Jim Lawrence]

Hi All:

I have been playing with the Windows 10, windows subsystem for Linux (WSL). Installed the latest Ubuntu version from the play store and then ran a backup from the Linux instance:

https://www.windowscentral.com/how-install-bash-shell-command-line-windows-10
http://bit.ly/2JIcuZm

It takes two lines of code to start the backup. My wife has thousands of pictures...just the indexes alone take 200+ GB:

First, connect to the network backup server:

sudo mount -t drvfs '\\BACKUPSERVER\10_10TB-Volume\10TB_Downloads\Pictures\MariasPictures' /mnt/MariasPictures

Note: the use of "drvfs". This is Microsoft's file system module which interfaces with any of the major possible 20 plus Linux filesystems (There are over 100 possible FSs but some are very rare). Also, in all Linux systems connections between other systems is done through a mount point. Example: "/mnt/MariasPictures" and Microsoft initiated a mount point for all the local drives. Example "/mnt/c", "/mnt/d", "/mnt/e" and so on.      

Second, start the backup. This command will transfer files anywhere in the world, with the proper syntax. Anywhere on the network is of course super quick. It is easy to forget and try and transfer a TB of data across the internet. ;-) Of course, as long as you can have a computer trickle transferring for a week or two. (Don't do this on Windws 10 as it will not multi-task very well and eventually the system will lockup.) ...how does he know you may ask... ;-)

rsync -avxHAX --info=progress2 /mnt/e/mlawrence/Pictures/ /mnt/Pictures 

If you do what to tranfer files across the internet here is an example of the code. Note: this example assumes you have an SSH client locally and a SSH server remotely. (On all Microsoft's new Windows 10 versions SSH is built in.):

rsync -avz --info=progress2 ~/Pictures/newpictures/* -e "ssh -p 18665" jimlawrence at 22.34.56.78:~/MariasPictures/newpictures/

Note: the above code copies all the pictures on your local computer, given the directory to a remote computer, given the appropriate directory and will prompt the remote computer's username and password (of course, just your appropriate SSH key can be used but that is another subject) and will do full compression. The assumptions are that the remote (home) computer is on, connected to the internet, has an SSH server running on it, has an appropriate port set up (in this case 18665 but it can be any value from 1 to 65535, in theory, with numerous exceptions, but the values tend to be used to elude hacker random port scanning, the SSH server must have the chosen listening port added and the network's router must be setup to redirect calls received on the particular port to the waiting computer.

If the rsync remote connection needs to be encrypted and use an SSH key instead of a user name and password combination so the data transfer process can be ultimately secure and completely automated, check out the following article:
https://www.digitalocean.com/community/tutorials/how-to-copy-files-with-rsync-over-ssh
https://do.co/2EGi7Ud

You can get your ISP's IP value from a dozen different way. Then you can replace the fake "22.34.56.78" with an appropriate value. The following link works fine for finding your host IP: https://www.askapache.com/online-tools/whoami/

If your Windows 10 computer does not have an SSH server installed check out the following: 
https://blogs.msdn.microsoft.com/powershell/2017/12/15/using-the-openssh-beta-in-windows-10-fall-creators-update-and-windows-server-1709/
http://bit.ly/2GSNq46

Router ports explained: 
https://en.wikipedia.org/wiki/List_of_TCP_and_UDP_port_numbers
http://bit.ly/1L2GEW1

To set your router pass-through porting check out your router's advanced features. It is very router specific; Lynxus, Netgear, Cisco etc.

Here is a list of all rsync options and what they mean: https://ss64.com/bash/rsync_options.html

Why is rsync probably the best universal backup/data transfer tool, ever created, at an price (free)? 
https://www.networkworld.com/article/3039839/linux/synching-your-teeth-into-rsync.html
http://bit.ly/2JHWBSm

In summary, just having access to the rsync command is reason enough to install WSL, on your Windows box. It is the simplest backup system and it is impossible to break and it never corrupts...if it doesn't work it is a syntax error or a hardware failure.

I hope someone finds this interesting. I have setup a number of sites with this combination and it has always worked well. For anyone needing some help in setting up your own system, just contact me offline. :-)

Jim